Sports scorekeeping system with integrated scoreboard and automatic entertainment system

ABSTRACT

An integrated sports scorekeeping and entertainment system ( 101, 102, 103 ). The system accepts game information from manually operated scorekeeping devices ( 100 - 4   c,    100 - 4   d,    100 - 8, 100 - 7 ) and automatic performance content generation sub-systems ( 201 ), for combining into either control commands and/or information that outputs to scoreboards ( 100 - 3 ), primary lighting systems ( 100 -L and  400 -J 8   a ), secondary lighting systems ( 400 -J 8   b ), music systems ( 400 -J 5 ), video systems ( 400 -J 6 ), goal score indicators ( 400 -J 7 ), dynamic advertising displays ( 400 -J 9 ), etc. The system ( 101, 102, 103 ) includes a universal scoreboard interface ( 100 - 6 ) that inputs pertinent game information in the form of a virtual scoreboard for translation into the physical signals for controlling the scoreboard ( 100 - 3 ), without the need or use of the Scoreboard&#39;s native control console ( 100 - 2 ).

RELATED APPLICATIONS

This application claims the benefit of priority to U.S. ProvisionalPatent Application Ser. No. 61/399,167, filed on Jul. 8, 2010, and, as acontinuation-in-part, to International Patent Application NumberPCT/US2009/056805, filed on Sep. 14, 2009 entitled SESSION AUTOMATEDRECORDING TOGETHER WITH RULES BASED INDEXING, ANALYSIS AND EXPRESSION OFCONTENT (herein also referred to as SARTRIA), which in turn claimspriority to U.S. Provisional Patent Application Ser. No. 61/192,034,filed on Sep. 15, 2008. The contents of all of the priority applicationslisted above are incorporated in this application by reference.

The present invention is also related to the prior inventions disclosedin U.S. Pat. No. 6,567,116 entitled MULTIPLE OBJECT TRACKING SYSTEM andU.S. Pat. No. 7,483,049 entitled OPTIMIZATIONS FOR LIVE EVENT,REAL-TIME, 3D OBJECT TRACKING.

FIELD OF INVENTION

The present invention relates to systems for automatically controlling asports scoreboard, music and announcement systems, video displays,scoring indication lamps, primary arena lighting systems, laser show andsecondary lighting systems, dynamic advertising display boards and otherevent entertainment devices.

BACKGROUND AND SUMMARY OF THE INVENTION

There are a large number of sporting fields where scoreboards are usedto display official game information to the teams and spectators, suchas in the sports of ice hockey, football, basketball, baseball andothers. Especially when these fields are used for amateur sports, thescoreboards carry basic information such as the official game time,period of play and team scores. There are several manufacturers of thesetypes of amateur sport scoreboards most notably Daktronics, Inc., thelargest supplier to the youth market.

Scoreboard systems from manufacturers such as Daktronics typicallyinclude at least two parts, the scoreboard itself as well as a consolefor remotely operating the scoreboard. The typical console has a keypadinterface for the console operator, an internal processor fortranslating the operator's indications into scoreboard changes, and oneor more output ports for transmitting these changes in some format toone or more scoreboards. There are several problems with this overallarrangement including the lack of interface with a third-party systemscorekeeping system, where such a system might be used for creatingcontent regarding the sporting event itself.

In prior applications, especially including the prior related SARTRIAapplication, the present inventors taught several aspects of suchcontent generating systems, mostly using the example of ice hockey. Thetypes of content created include recorded video and audio along withofficial box score information. The present inventors have also taughtseveral variations of using the same recorded video captured as content,to support tracking of the players and game objects via image analysis.When synchronized and cross-indexed, the video, audio, box score andtracking data have significantly greater use and value.

Such systems for creating content as taught by the present inventorsstill require that an individual operate the game scoreboard. Hence,even though these systems gather the official game scoring information,or box score, they do not also operate the scoreboard. The main reasonfor this is that manufacturers such as Daktronics tend to keep theirscoreboard console “closed,” not allowing for third-part equipment to beelectronically interfaced.

What is needed is a universal scoreboard interface module by which anymanufacturer's or most manufactures' scoreboard consoles or scoreboardscan be interfaced, so that any third party content generation system maydirectly control the scoreboard, thus removing the need for thescoreboard console operator.

In the present invention, two variations are taught of a firstembodiment—one for interfacing directly to the console, simulating theuse of the keyboard by a console operator, and the second forinterfacing directly to the scoreboard, simulating the control signalssent by the console. Regardless of the interface point, the presentinvention first comprises a training mode whereby an operator trains theinterface device regarding the appropriate signals for mimicking eitherthe keyboard signals input to the console, or the resulting controlsignals sent to the scoreboard by the console. This training may eitherbe fully automatic, or simply the recording of appropriate signals to beanalyzed by a programmer of the interface device. Once analyzed, theprogrammer will then establish the necessary data within the interfacedevice so that in its live operation, it will send identical signals toeither the manufacturer's console (thus mimicking the keypad entries) orto the manufacturer's scoreboard itself (thus mimicking the console).

Once trained, the interface module may then be connected to a thirdparty scorekeeping system which in turn then supplies the necessaryhigh-level commands such as “set clock to 99:99,” “start clock,” “stopclock,” “enter penalty . . . ,” etc., as the sporting event isconducted. In this live mode, for each supplied high-level command theinterface module then generates the equivalent keyboard signals forinput to the console, or resulting control signals for direct input tothe scoreboard.

In a second alternate embodiment, the universal scoreboard interfacedoes not require a training mode and simply connects directly to thescoreboard thus completely by-passing the manufacturer's console. Inthis case, the third party scorekeeping system maintains an internalvirtual scoreboard which the interface device monitors for output to thescoreboard. As will be shown, the ideal output format conforms to therequirements of the scoreboard and at least for manufacturers such asDaktronics, therefore includes signals for setting the individual cellsegment values for each displayed character on the scoreboard.

Using either variation of the universal scoreboard interface eliminatesthe need to operate the scoreboard manufacturer's console, which at theyouth sports level often means the reduction of at least one job (notethat a person is still required to operate the third party scorekeepingconsole). The present inventors will herein teach an additionalpreferred embodiment to a scorekeeping system that combines the use of aperformance content generation system to ultimately also eliminate thisneed for an operator of the third party scorekeeping console. As will beshown, using the aforementioned universal scoreboard interface the mainfunctions of a scorekeeping console are to set, reset, start and stopthe official clock and to enter the remaining official scorekeepinginformation, which in ice hockey for example includes data capturedwhile the game is in progress (shot counts by team) and data capturedwhile the game is halted (such as goals and penalties). The onlyremaining information entered by the scorekeeper is non-official andtherefore to some extent optional.

In this “scorekeeperless” configuration of the scorekeeping system, theofficial clock is maintained by a combination of a game official and theperformance content generation system, which itself includes some formof an object tracking system. (While other technologies such as RF, IR,UWB, GPS, etc. may be used for tracking the performance activities andstill accomplish the herein taught improvements, the present inventorsprefer and depict a camera-based object tracking system for followingthe player, referee and game object movements.) As will be shown, usingthe object tracking system the moment of puck-drop is automaticallydetectable for starting the clock and a game official can at least use awireless clicker instead of a whistle to stop play and the clock. Itwill also be shown that once the official presses the “stop-play”clicker button, the system can automatically generate the customarywhistle sound for notifying the players, team benches and fans. Thosefamiliar with referee whistle technology will understand that variationsare possible since technology already exists for automatically detectingthe whistle's sound waves.

The present invention will then also teach that the “scorekeeperless”system employs through-the-glass touch technology to face the “game-playhalted” data entry screen towards the game official. Doing this allows areferee to directly enter all official information such as goals andpenalties, as opposed to the traditional practice of speaking to thescorekeeper who then appropriately records the information eitherelectronically or on paper. And finally, at least for the sport of icehockey, the only other official information is the shot count by teamthat has at least two ways of being collected without requiring ascorekeeper present in the traditional scorekeeper's booth. Thepreferred method is to automatically track shots using the performancecontent generation system via its object tracking system, techniquesthat are both well understood and in some sports such as soccer alreadyimplemented. The alternate technique is to allow an operator to use awireless or internet linked portable tablet or similar device toindicate shot counts from any desired location, e.g. from the stands. Aswill also be taught, this operator can additionally use this remote dataentry solution for optionally entering desirable non-official gameinformation, e.g., player shifts, hits, face-off locations, etc.However, most of this additional non-official game information may alsobe automatically determined using a performance content generationsystem, especially and preferably as taught by the present inventors inthe prior SARTRIA application.

In addition to allowing a third-party scorekeeping system toautomatically and directly control a given manufacturer's scoreboard, itis also desirable to automatically control other devices such as themusic and announcement system, video displays, scoring indication lamps,primary arena lighting systems, laser show and secondary lightingsystems as well as dynamic advertising display boards. In general, thedesired other devices to control are usable for enhancing the enjoymentof the performance by spectators and, as will be discussed in theconclusion of this application, other types of “entertainment devices”exist and can also be automatically controlled using the herein taughtsystem. What will be taught is the combining of the scorekeeping systemand the performance content generation system for the enabling of anautomatic entertainment system, which then controls any number ofconnected entertainment devices such as listed above.

In the present inventors' prior patents and applications, there wastaught the integration of a real-time multiple object tracking system,preferably based upon cameras and machine vision, that was capable ofcapturing game video and audio, player and referee movements, wirelessclicker signals, as well as any and all other real-time human or machineobservations. In these applications, a universal protocol was alsotaught for encoding any human or machine observation into “marks” of agiven type, with a specific time of observation and optionally carryingrelated data further describing the observation. In turn, theseobservation marks where input into a session processor that had accessto external rules, where the rules directed how the incoming real-timemarks should potentially create, start or stop individual events of agiven type. These events then serve as a session index back into allrecorded continuous session content such video and audio.

It is now further herein taught that these same observation marks andevents may be translated by the session processor into a real-time datastream of automatic entertainment system (device) triggers. The actualtriggers output by the session processor are first filtered (or limited)based upon the specific types of entertainment devices to be commanded(e.g., a music system, announcement system, video display, etc.). Thepresent inventors prefer that these triggers as output by the sessionprocessor are accepted by a distinct automatic entertainment systemprocessor that is responsible for interfacing directly with any and allthird party devices. For each device type, the entertainment processorwill have access to a pre-known set of external trigger rules that canbe used to translate the incoming session content (i.e., detected marksand events) into a set of outgoing universal action commands for eachgiven device type. These universal action commands are then accepted byindividual device wrappers that perform a final translation of theuniversal command into one or more custom application programminginterface (API) sequences that are pre-established to cause specificresulting actions by the unique physical device.

The present inventors prefer this distinct multi-step translation fromagnostic session content into streams of session triggers filtered byentertainment device type, which in turn are convertible into universalaction commands for the given device type via a set of external triggerrules, which are then finally translated into custom (API) sequences fordriving individual unique third-party entertainment devices. Thiscombination of double abstraction and objectification allows theconnection chain from the session processor, entertainment system andunique entertainment devices to remain loosely connected and thereforeboth open and easily distributable.

Within the forthcoming specification, the present inventors will provideexamples of specific game observations along with the resultingperformance activity (event) create/start/stop transitions and how thesetransitions appropriately and ultimately may be used to control variousentertainment device actions—such as starting and stopping music, makingannouncements, showing video replays, turning on goal lamps, dimmingarena lights, etc.

The present inventors will also review some relevant teachings of theprior SARTRIA application for a SESSION AUTOMATED RECORDING TOGETHERWITH RULES BASED INDEXING, ANALYSIS AND EXPRESSION OF CONTENT. Duringthis review of the prior taught apparatus and methods of a sessionprocessor, the present inventors will show that these same teachingswith one preferred modification are also applicable and preferred forthe implementation of the herein specified automatic entertainmentsystem processor.

Therefore, given the state-of-the-art in computer systems, FPGAs,microcontrollers, wireless clicker technology, through-the-glass touchpanels, data transmission protocols, and object tracking systems, it ispossible to create the preferred system for automatically interfacing athird-party content generation system (that at least also accepts theofficial scorekeeping data) with any or most existing scoreboardsystems. This preferred system can be further enhanced to eliminate theneed for a scorekeeper.

Given the state-of-the-art in real-time sports tracking, content captureand performance analysis systems, it is also possible to create thepreferred sports entertainment system for automatically interfacing toone or more entertainment devices, the control of which is ultimatelylinked to the game activities measured by the tracking system viaexternal rules that may be altered without needing to change theentertainment system itself.

Objects and Advantages

Therefore, the objects and advantages of the present invention includeproviding a universal interface module that can be connected to ascoreboard console either at the juncture between the console's keyboardand its internal processor, or between its internal processor and thescoreboard itself (via the console's output port). This interface moduleprovides both training and live modes. In the training mode, the moduleis capable of recording various signals that are either supplied by thekeyboard to the console, or by the console to the scoreboard,representing the entire range of possible low-level commands performedby the manufacturer's scoreboard console. In the live mode, the moduleis capable of receiving high-level commands from a third-partyscorekeeping console which are then translated into the equivalentlow-level commands and transmitted either to the scoreboard console viathe keyboard, as if they were being directly entered by the consoleoperator, or transmitted to the scoreboard, as if they were beinggenerated by the manufacturer's console.

It is still a further object and advantage of the present system thatthis interface module be alternately capable of connecting a third-partyscorekeeping console to a scoreboard without requiring a training mode.In this case, the scorekeeping console maintains an internal virtualscoreboard that the interface module translates into the necessarysignals for updating the real scoreboard.

Another object and advantage of the present system is to provide for ascorekeeping system that allows the game officials in combination with aperformance content generation system that comprises an object trackingsystem, to perform all official scorekeeping tasks including at leastthe operation of the game clock and the entry of shots, goals andpenalties information—thus providing the option of eliminating thetraditional scorekeeper.

The objects and advantages of the present invention further includeproviding apparatus and data translations methods for receiving humanand machine observations from the combination of a scorekeeping systemand a performance content generation system as related at least to asports performance such as a game, and then automatically determiningwhen and which connected entertainment devices should be commanded totake which specific actions; where the entertainment devices at leastinclude music and announcement systems, video displays, scoringindication lamps, primary arena lighting systems, laser show andsecondary lighting systems as well as dynamic advertising displayboards.

And finally, the objects and advantages of the present invention includethe implementation of the preferred automatic entertainment processorusing the prior taught session processor and its various features.

As will be apparent to those familiar with the various marketplaces andtechnologies discussed herein, portions of the present invention areuseful individually or in lesser combinations than the entire scope ofthe aforementioned objects and advantages. Furthermore, while theapparatus and methods are exemplified with respect to the sport of icehockey, as will be obvious to the skilled reader, there are norestrictions on the application of the present teachings, whether toother sports, music, theatre, education, security, business, etc., andin general to any ongoing measurable activities, real, virtual,abstract, animate or inanimate, without limitation. The lack of a needor use in other such applications for a scorekeeping system does notreduce the benefits provided by using a performance content generationsystem in combination with an automatic entertainment system.

Still further objects and advantages of the present invention willbecome apparent from a consideration of the drawings and ensuingdescription.

DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts the current state-of-the-art in youth scorekeepingsystems, especially as they relate to the sport of ice hockey andwithout consideration of other recent advancements already taught by thepresent inventors in prior related patents and applications. The focusof the figure is the number of manual jobs that must be performed andthe uses for the data being created by those jobs.

FIG. 2 depicts the current state-of-the-art shown in FIG. 1 incombination with additional teachings of the present inventors fromprior patents and applications. Using these teachings, which include anobject tracking system for observing the game and an externalrules-based session processor for combining the game data into a sessionindex, the overall system eliminates two jobs while still increasing indata (session content).

FIG. 3 a shows a high level block diagram of the present invention'steaching for integrating the external scorekeeping system directly withthe scoreboard (that is normally controlled by the scoreboardmanufacturer's console). The integration is handled through a universalscoreboard interface combining both a novel digital circuit andsoftware. The interface includes both a training mode for learning themanufacturer's control signals as well as a live mode for mimickingthose signals in order to operate the scoreboard, or simply a live modecapable of driving a scoreboard without prior training.

FIG. 3 b is a high level block diagram that improves upon FIG. 3 a byshifting the job of operating the game clock to a combination of thereferee and the object tracking system, while also shifting the job forentering the official score sheet to the referee. This preferredembodiment of the scorekeeping system eliminates the scorekeeper's job,thus providing for a fully automatic scorekeeping system (outside of thesession participants such as the referee). Also depicted is the teachingthat allows (optional) human observations of game activities to be maderemote from the scorekeeper's booth and independently of the operationof both the scoreboard and the entering of official scorekeepingdata—functions that are traditionally all bundled together as thescorekeeper's job shown in FIG. 3 a.

FIG. 4 depicts a combination of the state-of-the-art along with thepresent inventors' prior improvement shown in FIG. 2, as well as the newimprovements shown in FIGS. 3 a and 3 b. The net result over the currentstate-of-the-art shown in FIG. 1 is the removal of all mandatory jobswhile at least maintaining traditional clock, scorekeeping andvideo/audio data, as well as their traditional uses.

FIG. 5 depicts all of the combined teachings shown in FIG. 4 with theinclusion of additional teachings for an automatic entertainment systemdriven by the session content comprising information from both thescorekeeping system and the performance content generation system. Thisautomatic entertainment system address several other manual jobs listedin FIG. 1 but not depicted, including making audio announcements andoperating the music system, choosing and driving the video display,operating the goal lamps and creating various lighting effects bydimming the venue lighting while also operating a laser projector.

FIG. 6 a is a mid-level block diagram specifically addressing the designof the universal scoreboard interface. In this first embodiment, theinterface module has a first training mode and .a second live mode. Inthe training mode, it records either keyboard or console processoroutput signals via either of two capture points in response to thecontrolled execution of an exemplary set of console commands enteredthrough the keyboard by the scoreboard console operator. In the livemode, these recorded signals (or variations thereof) are played backthrough the original capture point in response to scoreboard changecommands issued through a third-party scorekeeper's console in responseto its operator, thus making the signals appear as if they wereoriginated within the scoreboard console without requiring a scoreboardconsole operator.

FIG. 6 b is a mid-level block diagram showing a second alternateembodiment of the universal scoreboard interface. As opposed to FIG. 6a, the preferred interface module only has a live mode and thereforeonly needs to interface to the scoreboard and never to the scoreboardconsole (i.e., for “recording signals”). In this embodiment, thescorekeeper's console is enhanced to maintain an internal virtualscoreboard which is periodically monitored by a cell display programthat converts each character on the virtual scoreboard into theappropriate cell segments for driving the corresponding scoreboardcharacters.

FIG. 6 c is a block diagram of one possible implementation of the secondembodiment of the universal scoreboard interface as taught in relationto FIG. 6 b.

FIG. 7 a is a block diagram depicting the translation, as directed bypre-known external rules, of the real-time stream of human and machinesession observation (marks) into events (i.e., performance activities oflimited duration) forming the session index—all of which were taught bythe present inventors in a prior application. Further shown is how theseobservation marks, resulting events and session index, collectivelyforming a portion of the session content, are then additionally filteredand processed into a real-time stream of event changes (i.e., on/offtransitions) with related data. This translation of data then serves asthe input to the automatic entertainment system, referred to as the AESSession Content.

FIG. 7 b is a block diagram depicting the translation, as directed bypre-known external trigger rules, of the real-time stream of AES SessionContent into entertainment device state changes and universal actioncommands, where a device is for example some form of lighting control,laser projector, audio output, video output, goal lamp, dynamic boarddisplay, or similar entertainment or information output device. Furtherdepicted is the translation of the universal action commands into astream of custom application programming interface (API) sequencescapable of altering a specific device's state.

FIG. 7 c is a table showing the preferred data format of a universalaction command.

FIG. 7 d is a table showing exemplary entertainment device types, theirgeneral purposes along with potential data sources and example dataoutputs.

FIGS. 8 a, 8 b, 8 c, and 8 d are all taken directly from the presentinventors' prior related application entitled SESSION AUTOMATEDRECORDING TOGETHER WITH RULES BASED INDEXING, ANALYSIS AND EXPRESSION OFCONTENT (SARTRIA) with one new teaching shown in FIG. 8 d. Collectively,these figures will show that the prior taught (and herein depicted)“session processor” may alternately or additionally be used as theherein depicted automatic entertainment system processor.

FIG. 8 a was taken from the original SARTRIA FIG. 7 and gives anoverview of the flow and subsequent processing of information; startingwith the differentiation of external observation marks followed by theirintegration into events, synthesis into summary and tertiary marks(internal observations) and finally the expression of combinations ofthis collected internal session knowledge into various usable data. Thesteps of integration, synthesis and expression were preferably conductedby the session processor. As will be discussed, this same sessionprocessor using its original construction with one modification(depicted in relation to FIG. 8 d) can perform the desired functionsherein taught for the automatic entertainment system.

FIG. 8 b was taken from the original SARTRIA FIG. 24 a and is a nodediagram showing the associations between a create, start and stop mark(observation) and an event, each governed by a rule, all of which ispertinent to the step of integration. As will be discussed, event typesmay be used to represent individual automatic entertainment device typessuch that individual event type instances may then also serve torepresent individual entertainment devices and their current devicestates. It will also be shown that the rules governing this step ofintegration are adequate to serve as the herein taught external triggerrules for controlling the activation and deactivation of individualentertainment devices.

FIG. 8 c was taken from the original SARTRIA FIG. 31 and is acombination node diagram with a corresponding block diagram detailingthe relationship between the mark and event objects for creating andspecifying “tertiary” (“calculation”) marks. As will be herein taught,these same tertiary marks may be used to represent universal actioncommands for controlling the automatic entertainment devices.

FIG. 8 d was taken from the original SARTRIA FIG. 33 and is acombination node diagram with a corresponding block diagram detailing aspecial type of rule called a “descriptor,” originally used for eventnaming. As will be herein taught, these same descriptor rules may beadditionally linked to the related datum of (primary, secondary or)tertiary marks. This connection is established in template form byassociating a descriptor rule stack to a context datum (which itself isrelated to a mark—see see FIG. 8 c). By so doing, these same priortaught descriptor rules are equally useful for creating officialannouncement text, game commentary data streams and database SQL selectstatements, all of which are important features of the preferreduniversal action command.

DETAILED SPECIFICATION

Referring to FIG. 1 (Prior Art) there is depicted the currentstate-of-the-art sports scorekeeping system as it applies to youth icehockey. As will be understood by those familiar with sports in general,the arrangements and depictions in FIG. 1 and therefore carried into theremaining figures are directly applicable (with minor variations) to atleast all teams sports, especially those that include a scoreboarddisplay. Therefore, while the present invention is described in relationto the sport of ice hockey, this choice of exemplification should not beconstrued as a limitation. As will be understood by those skilled in thevarious arts supporting the herein teachings, the present embodiments asthey apply to the integrated scoreboard have applicability to sports ingeneral and also have uses beyond sports, such as in any settingincluding a performance, an audience and an electronic display.Likewise, the herein teachings as they apply to any one or more aspectsof the automatic entertainment system also apply to sports in general aswell as any setting including activities where the people conducting theactivities and/or watching the activities desire to receive the types ofinformation so provided.

Still referring to FIG. 1, the arrangement depicted includes aperformance area 100-1 where players such as 100-1 b interact while gameofficials such as 100-1 a make judgments and have effect over thestarting and stopping of game play. Also shown is a scoreboard 100-3used to post information regarding the performance along with ascoreboard console 100-2, typically provided by the scoreboardmanufacturer, to control the displayed values (data use 1) on thescoreboard 100-3 (herein referred to as “job 1”). (Note that examplescoreboard and console manufacturers would include Daktronics,Fair-play, Varsity Scoreboards and several others.) While most youthsporting events still keep the official box score on paper as the gameis played, the current state-of-the-art includes electronic scorekeepingsystems 100-4 a that are typically provided by some third-partymanufacturer (e.g., Pointstreak). Scorekeeping systems 100-4 a such asprovided by Pointstreak do not integrate with the manufacturer's consoleor scoreboard, either for the purposes of getting or giving information,but rather serve to record official box score information in real-time(herein referred to as “job 2”) which is then also ideally posted to oneor more web-sites on the internet (data use 2). Altogether, thecombination of performance area 100-1, console 100-2, scoreboard 100-3and scorekeeping console 100-4 a form an official scorekeeping system100.

Also depicted in FIG. 1 is a video camera tripod combination 200-1 aoperated manually as “job 3” for the recording of the game activities.Service providers such as Fast Hockey or B2TV provide web-boxes 200-2 awhich can directly accept the video stream from camera 200-1 a forstreaming onto the internet (data use 3). Note that these serviceproviders and their web-boxes 200-2 a only provide manually controlledvideo and do not also interact with the scorekeeping console 100-4 a orthe scoreboard 100-3 and its console 100-2, such as would be necessaryfor obtaining useful game time and box score information for graphicallyoverlaying onto the video. A separate job 4 is also shown where anoperator using video editing workstation 200-3 creates a video index forthe video and otherwise performs clipping and assembly as will beunderstood by those familiar with sports editing software such asprovided by XOS Tech, StevaSports, Gamebreaker and many others. The datacreated by this job 4 is typically made available to the team for grouppresentations (data use 4) or provided over the web for general publicviewing (data use 5). Altogether, the combination of camera tripodcombination 200-1, web-box 200-2 a and video editing workstation 200-3form a performance content generation system 200.

And finally, also depicted in FIG. 1 is a list of the other “Manual JobsNot Depicted” 100-a in addition to Jobs 1 through Job 4 that might beperformed at either a youth sporting event (such as Job 5 to makeofficial scorekeeping announcements and operate the music system) or ata professional event (such as Job 6 to operate a video display forexample to show instant replays of game action, Job 7 to operate a goallamp as is typical in ice hockey to indicate that either a home or awaygoal was scored, Job 8 to dim venue lighting and/or generate a laserlight show to build crowd excitement and Job 9 for controlling thecontent and effects used by a dynamic advertising system—all of whichwill be discussed in more detail with respect to upcoming FIG. 5).

Referring next to FIG. 2, the current state-of-the-art arrangementsfirst depicted in FIG. 1 are now augmented to include the improvementsalready taught by the present inventors in past related patents andpending applications. Specifically, the official scorekeeping system 100of FIG. 1 has been upgraded to system 101 in FIG. 2 and now includesscoreboard differentiator 100-5 for remotely capturing the face ofscoreboard 100-3 with a video camera, whereupon the images are thenanalyzed (differentiated) in real-time in order to ascertain the Data C(Use 1) therein contained. Hence, the data entered via scoreboardcontrolling console 100-2 as Job 1 are transmitted for display onscoreboard 100-3 at least including the state of the game clock (DataC). By using a camera to remotely detect the current state of scoreboard100-3, the scoreboard differentiator 100-5 at least informs the upgradedthird-party scorekeeping console 100-4 b of the changes in game-statefrom “in-play” to “out-of-play,” which in turn are useful forautomatically alternating the data entry screens used to prompt theoperator performing the scorekeeping Job 2. Many other importantfunctions for this Use 1 Data C were also prior taught, such as alwaysknowing the current period and game time which is critical for thecreation of a real-time game index. (Note that in the prior relatedapplications, a direct link was also discussed whereby the scorekeepingsystem 100-4 b could receive digital information as generated by thescoreboard console 100-2 equivalent fully or in-part to that informationdetermined remotely by differentiator 100-5 via scoreboard 100-3. Theadvantages of using the remote differentiator 100-5 are its remoteaspect as well as its inherent independence from the scoreboardmanufacturer's system—all of which were fully discussed in the priorapplications and will be well understood by a careful reading of thoseteachings.)

Still referring to FIG. 2 and also as already taught in the presentinventors' related applications, camera and tripod combination 200-1 ahas been replaced by camera and electromechanical pan-tilt device withcontroller 200-1 b while web-box 200-2 a for converting video has beenreplaced by compressor 200-2 b. Furthermore, performance area 100-1 isnow shown to be in the constant view of object tracking system 200-3that employs multiple fixed cameras with associated machine vision tofollow the activities of players such as 100-1 b and game officials suchas 100-1 a. These tracked activity motions are then preferably passed toperformance differentiator 200-4 that is responsible for determiningwhen selected activities exceed one or more threshold conditions, thusresulting in the issuing of “machine observations” 200-4 m as timeline“marks” with “related data.” Such observations might include “shiftchange,” “zone change,” “shot taken,” etc. Also fully taught in priorissued patents from the present inventors is that the activity (objectmotion) information generated by the object tracking system may be usedin real-time to operate the pan, tilt and/or zoom aspects of any one ormore side-view perspective cameras, such as 200-1 b—thus eliminating Job3 of operating the camera and tripod 200-1 a.

Upgraded scorekeeping console 100-4 b of FIG. 2 is also now capable ofissuing “human observations” 100-4 m as made by the scorekeeperperforming Job 2 in the same prior taught and preferred (but notnecessary) “mark” plus “related data” protocol for packaging machineobservations, such as those generated by performance differentiator200-4. Human observations made by the scorekeeper in combination withthe scoreboard differentiator 100-5 (and thus technically a machineobservation) might include “clock started,” “hit,” “shot taken,” “clockstopped,” “goal scored,” etc. The real-time observation streams of 100-4m and 200-4 m are then preferably made available to a session processor200-5, where processor 200-5 uses pre-established external rules 200-5 rto control the translation of all observation streams such as 100-4 mand 200-4 m into a session index and other content for storage insession content database 200-5 c. As previously mentioned, all of thishas been taught in detail by the present inventors in prior relatedapplications including that this translation of the human and machineobservation “marks” into performance “events” forms a natural “sessionindex,” which in turn is relatable in real-time to all captured video(and audio) content coming from either the fixed cameras of the objecttracking system 200-3, or the automated cameras 200-1 b.

In summary, current state-of-the-art official scorekeeping system 100and performance content generation system 200 depicted in FIG. 1 havenow been upgraded to scorekeeping system 101 and content generationsystem 201 using the teachings of the present inventors from priorrelated applications. This combination of upgraded scorekeeping system101 and content generation system 201 acts to eliminate the need for thevideo editing workstation 200-3 and its operator performing Job 4. Thusthe arrangement of current state-of-the-art devices shown in FIG. 1 isseen to require at least 4 operators performing: Job 1 to operate thescoreboard and clock forming Data C, Job 2 to operate the third-partyscorekeeping device 100-4 a forming Data S, Job 3 to operate the cameraand tripod combination 200-1 a forming Data V and Job 4 to operate thevideo editing workstation forming Data I. As shown in FIG. 2, byemploying the enhancements prior taught by the present inventors, bothJobs 3 and 4 can be fully automated, thus greatly enhancing at leastUses 3, 4 and 5.

Still referring to FIG. 2, at least video (with optionally capturedaudio) content Data V and session index content Data I are output tosession content database 200-5 c. Preferably, also output to database200-5 c are codifications of scorekeeping (box score) Data S andscorekeeper observation Data O—the codification by session processor200-5 of which was a subject of prior related applications (and notnecessary for the novel teachings of the present invention). Alsodepicted in FIG. 2 is session content publishing center 300-1 thateither automatically or with input from operators or purchasers,distributes any and all session content such as Data V, I, C and Seither locally or remotely (altogether referred to as session contentpublishing system 300).

Referring next to FIG. 3 a, the new teachings of the present inventionbegin with the addition of the first embodiment of a universalscoreboard interface 100-6 module to the upgraded scorekeeping system101 of FIG. 2. The overall purpose of this interface 100-6 is to shiftthe control of the scoreboard 100-3 from the manufacturer's console100-2 to the third-party scorekeeping console, now upgraded to 100-4 c.In prior scorekeeping console 100-4 a of FIG. 1, the device was simplyused to record the box score and official scorekeeping information ascommunicated by the game officials 100-1 a to the console operatorperforming scorekeeping job 2. Console 100-4 a had no automaticinteraction with the scoreboard 100-3 or its console 100-2. Inscorekeeping console 100-4 b, the scoreboard 100-3 is still controlledby its console 100-2, but it could at least assist in the scorekeepingjob 2 by remotely transmitting data to scorekeeping console 100-4 b viathe scoreboard differentiator 100-5. As was previously mentioned, thepresent inventors have also anticipated in prior applications andherein, that manufacturer's console 100-2 could provide via directconnection (i.e., not through the scoreboard 100-3) at least the sameinformation to scorekeeping console 100-4 b as determined bydifferentiator 100-5. The main point is that scorekeeping console 100-4b is fundamentally different from is earlier versions 100-4 a because itreceives and uses information generated by the manufacturer'sscorekeeping system (i.e., console 100-2 in combination with scoreboard100-3) in a novel and beneficial way for the benefit of the scorekeeper.

Still referring to FIG. 3 a, what is different about officialscorekeeping system 102 versus system 101 portrayed in FIG. 2 is thatsystem 102 uses scorekeeping console 100-4 c that is now driving thedata displayed on scoreboard 100-3—as opposed to responding to the data,which was the case with scorekeeping console 100-b. As will beunderstood by those skilled in the art of scoreboard systems, this is afundamental shift that has significant benefits for the scorekeeper,ultimately leading to the potential for shifting the main game-time dataentry to the game officials as will be discussed shortly in reference toupcoming FIG. 3 b. In order to accomplish this shift, it is necessary tohave an apparatus for connecting scorekeeping console 100-4 c toscoreboard 100-3, either directly or through scoreboard console 100-2,which is the purpose and one of the two main functions of universalscoreboard interface 100-6. (As will be well understood by thosefamiliar with hardware and computer systems, the universal scoreboardinterface 100-6 could just as well be integrated into the scorekeepingconsole 100-4 c rather than being external to it. Thus the physicalrelationship between the scorekeeping console 100-4 c, the universalinterface 100-6 and even the scoreboard 100-3 is immaterial to theteachings herein and should not be construed as a limitation thereof.)

Regarding the functions of the first embodiment of interface 100-6,there are mainly two. The first is to connect the manufacturer's console100-2 to the scorekeeping console 100-4 c (or its equivalent proxy, suchas a PC) running the necessary “training mode” software. As will bediscussed further in relation to FIG. 6 a, this training mode connectionis ideally made at either of two distinct junctures, points “A” or “B,”where A is the point inside scoreboard console 100-2 between itskeyboard and internal processor and B is the point outside console 100-2where it is connectable to the scoreboard 100-3. Regardless of theconnection point A or B, in this training mode, what is most importantis to operate manufacturer's console 100-2 over a sufficiently exemplaryset of its possible inputs (via its keypad) while simultaneouslyrecording all signals generated by console 100-2 in response.

As will be understood by those familiar with such scorekeeping systems,the keypad inputs and corresponding commands would typically be: “setmain clock to 99:99,” “set home goal to 9,” “start clock,” “stop clock,”“set away penalty 1 to player 99, time 2:00,” etc. In the training mode,as the operator runs through this sufficient set of exemplary commands,the universal scoreboard interface 100-6 transmits the signals itreceives within or from the manufacturer's console 100-2 to thescorekeeping console 100-4 c (or its proxy). During this transmission,it is further taught that the signals themselves will be converted fromthe original signal format used by manufacturer's console 100-2, into adigital signal format more readily useful to a typical PC. For instance,via juncture B the manufacturer's console 100-2 is typically using acurrent-loop with embedded digital data as a transmission signal toscoreboard 100-3, whereas the ideal input to the scorekeeping console(PC) 100-4 c is serial data—all of which will be well understood bythose familiar with both electrical engineering in general, and thesemanufacturer's scorekeeping systems in particular. These teachings willalso be reviewed in greater detail with respect to upcoming FIG. 6 a.

In reference to FIG. 3 a and the training mode, what is currently mostimportant is to understand that the function of the first embodiment ofa universal scoreboard interface 100-6 is to receive output signals fromthe manufacturer's console 100-2 in its native electrical format andthen to translate this same signal into preferably a serial data formatfor transmission to a typical PC without otherwise disturbing theunderlying (typically digitally encoded) information. It is alsoimportant to understand that the method of operation of the universalscoreboard interface 100-6 in the training mode is to receive, translateand transmit each specific set of output signals in distinct relation tospecific corresponding sets of exemplary keypad inputs to manufacturer'sconsole 100-2. As will be understood by those skilled in software andinformation systems, in its training mode the universal scoreboardinterface 100-6 is acting as a means for mapping the distinct signalsets issued by the manufacturer's console 100-2 (from either juncture Aor B) resulting from the exemplary job 1 keypad entries, with thecorresponding functions being accomplished by those same entries, e.g.,“start clock” and “increment home goal count.”

Still referring to FIG. 3 a, once the third-party scorekeeping console100-4 c has been sufficiently trained regarding the exact manufacturer'sconsole 100-2 signals generated at either juncture A or B to effect theexemplary scoreboard changes, then the universal scoreboard interface100-6 may be used in somewhat of a reverse manner during the “livemode.” In the live mode, the interface 100-6 connects the scorekeepingconsole 100-4 c back through juncture A or B to the scoreboard 100-3. Aswas prior discussed, the scorekeeping console 100-4 c is nowfundamentally different in that it is the source of all officialscorekeeping information (as opposed to simply recording it), mostespecially including the game clock state, which used to be maintainedin the scoreboard manufacturer's console 100-2. As the source ofinformation, while this information changes (e.g., a goal is scored, apenalty transpires, the game starts and stops, etc.) it is necessarythat the scorekeeper's console 100-4 c transmits this information to thescoreboard 100-3 (directly through juncture B or indirectly throughjuncture A within the scoreboard console 100-2) in a signal formatexactly similar to the scoreboard manufacturer's console 100-2—thusmimicking console 100-2 and accomplishing use 1 of at least the clockdata as well as some of the score sheet data. (Again, more details forthis teaching will be provided in relation to upcoming FIG. 6 a and thenalso for a second embodiment in FIG. 6 b, and finally the actual circuitboard layout for implementing the second embodiment will be shown inFIG. 6 c.)

As will also be understood by those skilled in both electricalengineering and in device manufacturing, there are significant tradeoffsfor using juncture A versus B. For instance, while the keypad signals(juncture A) tend to be less sophisticated to mimic (as opposed to thescoreboard signals (juncture B) directly sent from the scoreboardconsole 100-2 to the scoreboard 100-3), physically connecting theuniversal interface 100-6 to the portion of the scoreboard console 100-2that accepts its keypad signals requires opening the console 100-2 andmaking a separate internal connection. Based at least upon thecompactness of the scoreboard console's 100-2 internal design, this canbe problematic. In practice, there also tends to be a greater variationin the way keypads are internally connected to the console's 100-2 mainmotherboard than there are variations between the way any given console100-2 connects from its external port to a input port on the matchingscoreboard 100-3. (This is especially true within a single manufacture'sfull line of scoreboard's 100-3 and their accompanying console's 100-2that all tend to transmit the same signal format between each other viaexternal connections while the consoles themselves have distinctlydifferent keypads, different internal console designs and differentinternal connectors between their keypads and their console 100-2motherboards.) Thus, regarding the first embodiment of the universalinterface 100-6, it is preferred that the signals learned in thetraining mode and mimicked in the live mode are those normally output bythe scorekeeper's console 100-2 for direct input to their scoreboard100-3 (as essentially depicted by the “training” and “live mode” signal-path arrangements shown in FIG. 3 a).

However, as will be discussed in detail with relation to FIG. 6 b, it ispossible to implement a second embodiment of the universal scoreboardinterface 100-6 that still (and only) interfaces through juncture B, butno longer requires a separate training mode. It is this embodiment thatthe present inventors have implemented, for which the actual circuitlayout is shown in FIG. 6 c.

As will be appreciated by those familiar with scorekeeping systems ingeneral, the major benefit of using the official scorekeeping system 102of FIG. 3 rather than systems 101 or 100, is that the job 1 of operatingthe manufacturer's console 100-2 during a live game has been eliminated.

Referring next to FIG. 3 b, official scorekeeping system 102 has nowbeen further upgraded to system 103 where the job 2 of entering clockand official scorekeeping data has been shifted from the scorekeeper tothe game official 100-1 a in combination with the object tracking system200-3. To accomplish this new preferred functionality, the third-partyscorekeeping system is now also upgraded to 100-4 d such that itincludes a wired connection to “through-the-glass” touch interface 100-8and a wireless connection to both referee clicker 100-7 and optionalgame observation entry device 100-2 a. Note that like scorekeepingdevice 100-4 b of FIG. 2, upgraded device 100-4 c also includes acritical connection to performance differentiator 200-4.

Still referring to FIG. 3 b, and specifically to how the various newparts of scorekeeping system 103 work together to control the scoreboard100-3, there are two key functions to consider. First, how does thesystem start and stop the scoreboard 100-3 clock without a scorekeepereffectively controlling the scoreboard 100-3 via the scorekeepingconsole 100-4 c? Within ice hockey, the clock is started under a strictset of rules including:

-   -   1. The game must be in an official period;    -   2. The clock must be currently stopped;    -   3. The players must be aligned for a face-off, and    -   4. The referee must drop the puck in the middle of this        alignment.        The data states supporting rules 1 and 2 are easily known to the        system by scorekeeping console 100-4 d as would be obvious. As        will be understood by those skilled in the art of machine vision        systems, the present inventors' prior art teachings for a camera        based object tracking system 200-3 provide at least one        sufficient way of detecting the data states supporting rules 3        and 4. It is not the purpose of the present invention to teach        the necessary image analysis algorithms for determining current        object (i.e., player and puck) locations and then subsequent        object movement (as the face-off begins), especially since the        underlying algorithms are already fairly well understood in the        art. What is important is that the data are collected by the        object tracking system 200-3 and ultimately passed to the        scorekeeping console 100-4 d. (Note that there is also no        restriction for the purposes of the present invention that the        object tracking system 200-3 be implemented with cameras and        machine vision—this is only the present inventors' preference.        For instance, as discussed in prior applications, use of RF or        IR systems is also acceptable for tracking at least 2D player        and puck locations, the location of which themselves are        sufficient for evaluating rules 3 and 4. Furthermore, for the        purposes of the present invention it is not a requirement that        the object tracking system 200-3 include a separate performance        differentiator 200-4. The system works equally well if the        object tracking system 200-3 provides the required data directly        to the scorekeeping console 100-4 d which does its own        differentiation, or includes an embedded differentiator; all as        will be well understood by those familiar with various software        systems and architectures.)

Still referring to FIG. 3 b, after the system uses performance contentgeneration system 201 to determine the “puck drop” game statetransition, the appropriate observation mark 200-4 m (or someequivalent) is transmitted to scorekeeping console 100-4 d that in turninitiates its own internal game clock. As this virtual clock isinitiated, its changing information is automatically conveyed throughthe universal scoreboard interface 100-6 to the scoreboard 100-3. It isnoted that typically the scoreboard 100-3 itself does not include aninternal clock that is turned on and off, but rather is simply used todisplay the cell values of the official clock traditionally maintainedin the manufacturer's console 100-2 and now maintained in thescorekeeper's console 100-4 d.

As will be obvious to those skilled in the various sports, onlyice-hockey starts its official scoreboard clock based upon the droppingof a puck. Some sports (like baseball) do not have an official clock inthe same sense of limiting the overall time of play. Other sports suchas basketball start the clock when their game object (i.e., thebasketball) is thrown in the air by the game official while in stillother sports like football the clock is started when the game officialblows their whistle. For the purposes of the present invention, theparticular method is immaterial. Again, what is important is that this“start clock” state is detected preferably by direct machine observation(or some other object tracking technology) and/or by allowing the gameofficial to so indicate using a wireless clicker 100-7 or someequivalent (such as blowing a whistle to activate either an airflowdetector or a sound receiver, both methods of which have been taught inthe prior art).

Once the game clock is started, and still referring to FIG. 3 b, in thesport of ice hockey (and several others) the clock is stopped when thegame official 100-1 a blows his whistle. While the present inventorshave already taught a whistle capable of detecting air-flow through itsinner chamber as a means of determining the point of whistle blowing,and other inventors have taught systems for detecting the sound createdby the whistle blowing, it is herein preferred that the game officials100-1 a be given wireless clickers 100-7. Regardless of the detectionapparatus and method, once the game official indicates that the clockshould be stopped, at least using the present wireless clicker 100-7 thescorekeeping console 100-4 d receives this “stop clock” signal via itswireless receiver. Similar to-the processing of the clock start signal,once the signal is received console 100-4 d stops its internal virtualclock which is then automatically reflected on scoreboard 100-3 viainterface 100-6. The present inventors also make note that the sound ofthe whistle is an important audible cue, not just to the scoreboardconsole operator to stop the clock, but also to the players to stoptheir activities. For this reason, the present inventors herein teachthat the integrated automatic entertainment system to be discussed inrelation to upcoming FIGS. 5, 7 a and 7 b, which among other thingscontrols the venue sound system, is to be signaled to generate awhistle-like or equivalently recognizable “stop play” sound.

While the scorekeeping system 103 of FIG. 3 b is shown to be able tostart and stop the game clock without the need of a scorekeeper, thenext data that must be addressed to fully eliminate this samescorekeeper at least includes the official entry of goals and penalties.While the current practice in ice hockey is that the game official 100-1a skates over to the scorekeeper and verbally indicates this informationfor data entry, the present inventors prefer allowing the game official100-1 a to enter this information by themselves. Past inventors havetaught a small data entry pad worn by the referee around their wrist orforearm with a wireless connection to the scorekeeping console. Whilethis solution is perfectly acceptable for the purposes of combining withthe present invention's start/stop detection (i.e., to eliminate theneed of a separate scorekeeper), the present inventors prefer notrequiring the referees to wear any equipment that may hurt them or getdamaged in the case of a fall.

There are presently several variations in the marketplace of a newtechnology branded as “through-the-glass” touch input screens, such asthose provided by PointandPress, VisualPlanet, iWindow and others. Theactual manufacturer or underlying technology is not important to thepresent invention. What is important is that using any of these devicesthe referee may enter official game information directly without theneed for a separate scorekeeper. Especially for the unique sport of icehockey that surrounds its performance area with boards and glass, any ofthe “through-the-glass” input screens would therefore be mounted on the“outside” of the glass opposing both the game official 100-1 a and moreimportantly the game action. Furthermore, all of these devices havesolutions that work through the thicker glass used in ice hockey and allcan produce screens of sufficient size to make entering the limitedinformation of goals and penalties simple for the official 100-1 a. Aswill be understood by those familiar with software systems, what isbeing accomplished is that the data entry screens typically used by thescorekeeper via console 100-4 c are now be presented to the gameofficial 100-1 a via the through-the-glass mounted data entry screen.Once the official Data S are entered, they are then used by scorekeepingconsole 100-4 c to update its internal virtual scoreboard at which pointthe same Data S are output to scoreboard 100-3 via universal interface100-6, also referred to as “use 1.” Scorekeeping console 100-4 c furthertransmits this same official Data S as observation marks 100-4 m (orsome equivalent) to session processor 200-5 (as depicted in FIG. 2).

As will be obvious to those skilled in the sport of ice hockey, there issome minor additional scorekeeping data beyond goal entry and penaltyentry that also needs to be taken care of by the game official 100-1 ausing this preferred approach. This Data S include penalty shots,overtime periods and shootouts but does not need to include the entry ofrosters, team names and dates. This later set of information will bepre-known to the scorekeeping system via its integration with acompanion scheduling system, all as taught by the present inventors inprior pending applications and not material to the present teachings.The present inventors also make note that the object tracking system200-3 and the performance differentiator 200-4 will be able to assistthe game official 100-1 a with some of the data entry, at leastincluding the jersey numbers and names of all current players (by team)on the ice, thus making a short list to pick from when entering goalsand penalties. Beyond this, the present inventors also anticipate thatthe differentiator 200-4 will be able to detect players entering thepenalty box as well as players scoring goals and providing assists, allof which can be used to simplify the required data entry. Regardless ofthe amount of assistance provided, the present invention offers theopportunity of eliminating the now separate job of scorekeeping by bothautomatically detecting the clock start/stop transition points as wellas accepting scorekeeping data directly from the game official 100-1 a.

And finally, still referring to FIG. 3 b, in the lower left hand corneran operator is depicted using game observation entry device 100-2 a toperform new optional “Job 2 o.” As will be understood by those familiarwith sports, there is a significant amount of additional “non-official”information that is still very desirable to collect in real-timesynchronization with all other clock and scorekeeping data. For icehockey, these other “observations” might include: “hits,” “icing,”“off-sides,” “highlights,” etc. What is preferred by the presentinventors is that this information may be entered remotely by virtuallyany observer using some networked device ranging from a tablet PC(connected via a private local wireless network) to a cell phone(connected via an internet web-page). Regardless of the device used ormethod of connection, what is herein taught is that the non-official,yet important other observations (Data O) are optionally enterable byone or more observers remotely (from the scorer's booth) and distinctlyfrom the clock (Data C) and scorekeeping (Data S). These devicesimplementing observation entry device 100-2 a also preferably, but notnecessarily, provide their information in the standard observation mark100-2 a-m protocol used by all other machine or human observationcollecting devices within the present system.

As will be well understood by those familiar with software systems, thenovel teaching of the present invention should not be limited by thechoice of information exchange formats or data transport mechanismsbetween the various system parts. In prior applications, the presentinventors taught the benefits of establishing a universal protocol andmethodology for collecting human and machine observations from adisparate range of devices to be collectively processed by a sessionprocessor. Such a universal approach allows for significant scaling ofdata collection devices as the marketplace adopts a single standard forsession content codification. This prior taught protocol extended beyondthe observation marks to include their processing into session eventsunder the control of externally provided rules. While these priorteachings of “marks, events and rules” are incorporated in the presentspecification and especially in relation to FIGS. 7 a, 8 a, 8 b, 8 c and8 d they are not necessary for the herein taught and claimed novelapparatus and methods, and should not be taken as limitations.

Referring next to FIG. 4, the entire combination of preferred officialscorekeeping system 103, performance content generation system 201 andcontent publishing system 300 is depicted together as a single systemwithout any additional new teachings.

Referring next to FIG. 5, there are shown systems 103, 201 and 300,exactly the same as in FIG. 4, with the addition of automaticentertainment system 400. Similar to content publishing center 300-1,entertainment system processor 400-1 is driven in real-time by sessioncontent database 200-5 c to be discussed in greater detail with respectto FIG. 7 a. Similar to session processor 200-5, entertainment processor400-1 employs external rules 400-1 r to govern its automatic functionsto be discussed in greater detail with respect to FIG. 7 b.

Still referring to FIG. 5, the purpose of the automatic entertainmentsystem 400 is to provide the other “manual jobs not depicted” 100-a asshown in FIG. 1 to a sporting venue in general, and in this example anice hockey facility. In its most abstracted view, entertainment system400 is responsible for either directly outputting portions of sessioncontent 200-5 c via one or more devices, where this output is typicallyaudible or visual, or effecting environmental changes in response tothis same content 200-5 c. In order to replicate the current manual jobsshown in 100-a, the entertainment devices taught in the presentinvention include:

-   -   Audio System 400-J5:        -   For performing Job 5—making announcements and playing music;    -   Video Display 400-J6:        -   For performing Job 6—operating the video display (e.g.,            showing replays);    -   Goal Lamp 400-J7:        -   For performing Job 7—operating the goal lamps when a goal is            scored;            -   (particular to ice hockey, although other sports will                play music or make a noise, which could be included with                Job 5 above);    -   Electro-mechanical Shutter 400-J8 a:        -   For performing Job 8—operating lighting effects by dimming            the facility lighting 100-L, especially in combination with,    -   Laser Projector 400-J8 b:        -   For performing Job 8—operating lighting effects by            projecting various laser patterns as a light show,            especially effective when the facility lighting has already            been dimmed, and    -   Dynamic Board Display 400-J9:        -   For performing Job 9—controlling dynamic advertising during            the game.

What is most important about automatic entertainment system 400 for mostsporting applications is not any one of these devices, but rather theoverall combination of such devices with at least an officialscorekeeping system 100, 101, 102 or 103 in combination with a contentgeneration system 200 or 201. Note that it is not necessary to have anassociated content publishing system 300 to apply the teachings of thepresent invention regarding entertainment system 400. Furthermore, thepresent inventors anticipate that additional devices might be desirablefor control by the entertainment system 400, especially for other sportsand other non-sport applications—all as will be known to a skilledsoftware systems engineer since an important aspect of system 400 is itsuniversality and open protocols. (Examples of other possibleentertainment devices include a “fogger” for creating fog over andaround the performance area or multi-color LED lighting for both castingwhite light during normal performance activity and switching to coloredlighting at appropriate non- activity times.) It is also noted that forsome sports and other types of non-sporting events, where a scoreboardis not used or desirable, the present teachings for using someimplementation of a performance content generation system 201 (ideallyincluding some form of object tracking) for creating machineobservations 200-4 m, and optionally using human observers on remotedata entry devices for creating human observations 100-4 m, are stillconsidered within the scope of the present invention.

Still referring to FIG. 5, the following list is provided of exemplarydevices found in the marketplace today that could serve the hereintaught purposes as entertainment devices:

-   -   Audio System 400-J5:        -   While many options exist, the present inventors prefer a            class of systems known as “Public Announcement (PA) over IP”            systems. Their common features include:            -   Connectable via standard computer networking                infrastructures (i.e., not centralized analog);            -   Easily driven by commands from a connected computer                system;            -   Work with pre-stored digitally recorded music;            -   Work with text-to-speech or pre-recorded speech;            -   Support optional live audio announcements; and            -   Use speakers with IP addresses capable of reproducing                analog audio signals from the transmitted digital                signals.        -   Example providers include:            -   Barix AG of Switzerland;            -   Kintronics Information Technology of NY, USA; and            -   Stentofon Communications of Australia.    -   Video Display 400-J6:        -   Many reasonable options exist, while the present inventors            prefer newer solutions such as:            -   Public-display-grade, “TileMatrix” LCD displays such as                the NEC V421 or V461. Each panel displays in 1080p                hi-definition and up to 25 panels can be combined to                form what is being referred to as a “video wall”;            -   Very large size displays such as the Panasonic 260 cm or                103 inch 1080p plasma, which also comes with a touch                version making it ideal for interactive use;            -   LED Display modules from manufacturers such as Toshiba,                Barco, Lighthouse, Daktronics and Element Labs. These                are basically smaller panels (often only 40 cm or 16                inch squares) that can be combined to make a                custom-shaped Video Wall that acts as a single display                screen; or            -   SMD LED Display strips from manufacturers such as                PowerPro from China. These are single lightweight LED                displays that can be manufactured into strips as large                as 50 m×        -   Many reasonable options exist, while the present inventors            prefer an arrangement of one or more USB LED Visual Signal            Indicators manufactured by Delcom Products, mounted behind            the glass, behind each goalie within a transparent casing.            Furthermore, using at least five signal indicators in            combination allows the system to automatically indicate the            location of the goal (e.g., “1—hole, 2—hole . . . 5—hole” by            turning on the matching number of lights). This information            is preferably determined automatically by the performance            differentiator 200-4 as it analyzes the data collected by            the object tracking system 200-3, all as will be fully            understood by a careful reading of the present inventors'            prior applications.            -   The present inventors further anticipate that this same                method of both indicating a goal, as well as some                characteristic of the goal, by using multiple LED lights                is applicable to other sports. For example in                basketball, either 2 or 3 goal lights could be lit based                upon the location the shot was taken from. Other                examples will be obvious based upon those familiar with                the various sports as all goals do have some                differentiating qualities that can be represented either                numerically (by the count of lights) or even by                different colored lights.        -   There are some manufacturers such as Daktronics that            specifically produce goal lights for ice hockey. Note that            these products use a red light to signify a goal and a green            light to signify the end of a period. As will be understood            by a careful reading of the present invention, turning on            the different colors for goal (red) or period-end (green)            are two separate commands to be issued to the Daktronics            goal lamp based upon the content generation system's 201            determination of either goal (which is preferably a machine            observation 200-4 m coming from performance differentiator            200-4, but could be a human observation coming through            scorekeeper's console 100-4 a, 100-4 b, 100-4 c or 100-4 d,)            or of a period-end (which is preferably an automatic            determination made through the scorekeeper's console 100-4            a, 100-4 b, 100-4 c or 100-4 d).    -   Electro-Mechanical Shutter 400-J8 a:        -   The present inventors prefer that the arena whose lighting            is to be controlled uses some form of LED lights rather than            the more traditional metal halide or fluorescent lighting.            For the present purposes of dimming, the main advantage of            LED lights is their ability to be cycled on and of at            differing rates, thus producing different lighting levels,            without damaging a ballast (i.e., the typical metal halide            lamp should not be “flickered”). Hence, with LED lights the            present system would send various commands that alter the            rate of on verses off cycles in order to create the dimmed            effect.            -   A further advantage of using LED lights is that they                often combine multiple individually colored LEDs (e.g.,                red, green and blue) that when powered in different                combinations can create up to 16,000 or more distinct                colors, including white which would be used during the                game. In addition to changing the brightness (a function                of luminosity) the present inventors further anticipate                creating various color effects by automatically sending                commands that change the mix of red, green or blue                individual LEDs that are powered on at a given moment.                One example use would be to flash the lights in the                color of the scoring team for each goal—and even to only                have those lights flash on either the side of the                performance area 100-1 where the goal was scored, or                perhaps over the bench area of the scoring team. Various                other ideas are possible as will be known to those                skilled in the various sports as well as the types of                characteristics that a given lighting system allows to                be controlled. What is important to note is that any                characteristic inherent to the lighting system that can                be controlled, is useable by the present invention and                triggerable by any machine 200-4 m or human observation                100-4 m, or combination thereof output as session                content 200-5 c.        -   However, especially in the amateur market the vast majority            of sporting venues still use some form of lighting that            cannot be easily dimmed by controlling its on-off cycle; for            example metal halide lamps. In this case, the present            inventors prefer using electronically controllable            mechanical shutters such as made by Wybron. The company            offers several variations and even advertises the use of            such dimmer/dousers in sporting arenas to create a dimming            effect. What is new with the present invention is that these            devices are automatically controlled, especially in            combination with all other devices herein exemplified. As            will be appreciated by those familiar with arena            entertainment responsibilities, the ability to pre-establish            rules for detecting any number of conditions instantly            triggerable by human or machine observations has significant            advantages, let alone the savings of labor expenses.    -   Laser Projector 400-J8 b:        -   For this device the present inventors have identified at            least one commercial product suitable for integration with            the automatic entertainment system 400 herein described.            Namely, the manufacturer ProlaserFX offers several products            they term as “programmable laser graphics projection            systems.” These devices are capable of receiving real-time            computer commands or executable scripts for controlling            their various features, including the ability to control the            laser image projected and to run various preprogrammed laser            show sequences.    -   Dynamic Board Display 400-J9:        -   Layer 2: This is the active video display layer for which            the present inventors prefer using such technologies as            amorphous silicon transistor (AST) panels which are now            being manufactured by Hewlett Packard. This or similar            technology is the preferred choice for use, mounted along            the inside (i.e., facing the action) of the ice hockey rink            boards because of their: flexibility (to fit the curved            board surfaces), large sizes (e.g., a typical board            advertisement is 234 cm×91 cm or 92″×36″), ruggedness (to            stand up to pucks and player contact), and low cost (roughly            $10 per square foot, which is significantly less than other            technologies).            -   Similar competitive displays (i.e., large area and                flexible) using variations of OLED technology are being                developed by several companies including Phillips, Sony,                Universal Display Corporation, and other manufacturers.        -   Layer 1 (optional): To support a brighter display, the            present inventors also prefer back-lighting the AST or OLED            panel with a LCE panel provided by CeeLite (or similar) that            operates on a variation of electroluminescence technology.            Like the AST array, CeeLite's panels are flexible and can be            made in custom sizes large enough to fit a 234 cm×91 cm (or            92″×36″) area.            -   Similar competitive back-lighting panels are being                manufactured with various OLED technologies including                from manufacturers such as Phillips and Lumiotec.        -   Layer 3 (optional): To augment ruggedness, the present            inventors anticipate placing a thin polycarbonate            (transparent) panel over the video display Layer 2,            something that can then also be replaced over time as it            becomes scuffed and scratched.            -   As an alternative, Zagg sells a nano-carbon based                invisible thin film that can be adhered to the Layer 2                display screen for scratch proofing and protection, as                opposed to the thicker polycarbonate solution.        -   Moisture proofing additive (optional): And finally, to avoid            the negative effects of moisture on the electronic            components, the present inventors prefer using a water            resistant coating on all appropriate Layer 1 and 2            components, or at least those portions of the components            that are susceptible to water damage. One such manufacturer            of water resistant coatings is Golden Shellback Technology            also marketed by ZAGG.

The aforementioned devices are exemplary for providing significantentertainment effects, especially for the example sport of ice hockey.Many or all of these devices can have the same, similar or differentuses for either other sports or other types of events, such as but notlimited to theatre, concerts, assemblies, conventions, corporatepresentations, entertainment parks, etc. Conversely, other similarproducts could be used to obtain the same features or sufficientfeatures.

What is most important is the teaching of a performance area 100-1 whereparticipant 100-1 a, 1 b actions are monitored for either human and/ormachine observations, the observations 100-4 m, 200-4 m respectively ofwhich are then 'combinable into content 200-5 c. Content 200-5 c thenserves as input to an automatic entertainment system 400 that operatesin real-time to translate this input into distinct electronic commandsfor controlling the operation of one or more entertainment devices, suchas but not limited to 400-J5, 400-J6, 400-J7, 400-J8 a, 400-J8 b and400-J9. What is further preferred but not necessary is that thetranslation of human and machine observations 100-4 m, 200-4 mrespectively are translated into content 200-5 c via external rules by aprogrammable session processor 200-5. What is also further preferred butnot necessary and to be discussed in detail with respect to upcomingFIGS. 7 a and 7 b, is that the translation of the content 200-5 c intodistinct electronic commands for controlling the operations of aneternal device is also effected by the automatic entertainment system400 under the control of external rules, therefore also making system400 programmable, similar to session processor 200-5.

As will be appreciated by those skilled in the art of software systemsand familiar with entertainment settings, many other devices may bedesirable for use with automatic entertainment system 400. Therefore,while the use of the present exemplary devices 400-J5, 400-J6, 400-J7,400-J8 a, 400-J8 b and 400-J9 is specifically disclosed herein, thesedevices are not to be construed as limiting the present invention. Forinstance, other devices are conceivable such as using a fog machine thatis used to create instant fog effects especially in the performance area100-1. Again, what is most important is that any device capable ofreceiving real-time electronic commands for at least one of its actionscan be used by the present invention's entertainment system 400 toexecute those actions in response to the observed, detected, sensed andotherwise known activities of the participants 100-1 a, 1 b in aperformance.

Referring next to FIG. 6 a, there is shown a mid-level block diagram ofthe universal scoreboard interface 100-6 as it interconnects with thescoreboard console 100-2, the scoreboard 100-3 and the scorekeeper'sconsole 100-4 c (or 100-4 d,) all of which was included in officialscorekeeping system 102 as first taught in relation to FIG. 3 a. Alsoshown in FIG. 6 a is the prior teaching of the present inventorsregarding scoreboard differentiator 100-5 as it remotely detects facechanges on scoreboard 100-3 for input into scorekeeper's console 100-4,all of which was first depicted in relation to FIG. 2. Briefly regardingscoreboard differentiator 100-5, its minimal purpose is to ascertain theofficial time of game and whether the game clock is running or stopped.As fully discussed in the prior SARTRIA application, this information isof significant value for forming the session index (Data I) related toall other session content 200-5 c including the video and the box score.Furthermore, also as prior taught the current states of the game clock,i.e., running or stopped, are important for automatically alternatingthe input screens on scorekeeping console 100-4 b (FIG. 2) to stayrelevant to the game situation. However, there is no new teachingproviding herein with respect to the differentiator 100-5 and itspresence is not required by design if the universal scoreboard interface100-6 is in use.

Still referring to FIG. 6 a, manufacturer's scoreboard console 100-2 isshown to internally comprise two main parts, namely keyboard 100-2 kwhose output connects to console processor 100-2 p input throughjuncture point A, 100-6A. Console 100-2 is also shown to have juncturepoint B, 100-6B, where the output of processor 100-2 p connects to theinput of one or more scoreboard(s) 100-3. The present invention teachesthat a first embodiment of the universal scoreboard interface 100-6 isconnectable to either console juncture A or B, 100-6A or 100-6B,respectively. In practice, it is anticipated that the interface 100-6would be implemented as connectable to A or B, rather than A and B.Regardless of the connection point A or B, Interface 100-6 should have atraining mode in which the scoreboard console 100-2 provides a series ofoutputs signals (i.e., from point A or B) in response to being operatedover a series of fully-representative input commands. Hence, in“training mode” the console 100-2 is operated normally via its keyboard100-2 k through a set of game-representative scoreboard commands, suchas:

-   -   Turn scoreboard on, off;    -   Set Game Period to value (9);    -   Set main clock to value (99:99:99:99);    -   Start/Stop clock;    -   Set Home, Away Goals to value (99);    -   Set Home, Away Shots to value (99); and    -   Set Home, Away Penalty 1 (or 2) to value—Player # (99) and        Penalty Duration (99:99).

While the above commands are typically all that are necessary foroperating the scoreboard at a youth ice hockey game, other sports willobviously have a different set of commands. What is important is that asthe training operator presses various keys on console keypad 100-2 k toindicate a command (where each command sequence typically ends with the“enter” key), the universal scoreboard interface 100-6, either connectedto juncture A or B, 100-6A or 100-6B, respectively, intercepts eitherthe keyboard's 100-2 k or the console processor's 100-2 p uniqueresponses for transmittal to a Command Translation Program 100-4-ctprunning either on the universal interface 100-6, the scorekeeper'sconsole 100-4 c, 100-4 d, or even a separate PC. As will be understoodby those skilled in software programming, the purpose of the CommandTranslation Program 100-4-ctp in training mode is to build a tableassociating a pre-known command (e.g., “start clock”) with itsequivalent A or B output signals. Preferably, the Command TranslationProgram 100-4-ctp stores this table of information in a database such asScoreboard Console's Command-to-Signal Recordings database 100-4-db.Thus, after all representative commands have been entered, they areessentially repeatable by way of Recordings database 100-4-db.

For instance and still referring to FIG. 6 a, if the scorekeepingconsole 100-4 c, 100-4 d is then connected in “live mode” to theuniversal scoreboard interface 100-6 which is further connected to theoriginal training capture point A or B, 100-6A or 100-6B, respectively,console 100-4 c is then able to recreate valid “scoreboard changecommands” which are oppositely translated by Command Translation Program100-4-ctp back into equivalent A or B juncture signals, for direct inputinto console processor 100-2 p (via juncture A) or directly toscoreboard 100-3 (via juncture B), respectively.

In practice, the most difficult command to train and record is theresetting of the game clock to any and every possible time value. Aswill be understood by those familiar with ice hockey, a typical amateurgame period may be anywhere from 12 to 20 minutes long and includes atleast 4 consecutive numerals such as: tens-of-minutes (TT), minutes(MM), seconds (SS) and tenths of a second (tt). In this case, theCommand Translation Program 100-4-ctp is preferably operated in trainingmode to capture a representative set of clock resets, covering the rangeof possible clock cell values. Since scoreboard console 100-2 uses thesame signals to represent a given number (i.e., “0” through “9”) for anygiven cell (i.e., TT, MM, SS or tt,) it is only necessary to have theCommand Translation Program 100-4-ctp record a single “T,” “M,” “S” or“t” cell cycling through each “0” to “9” value. As will be understood bythose skilled in the art, the Command Translation Program 100-4-ctp canthen automatically use the varying cell signals for each individualpossible cell value to recreate the necessary signals for setting anycell and all cells to any combination of possible values.

As will be known to those skilled in the understanding of devices suchas scoreboard console 100-2, having to physically connect to a juncturepoint A, 100-6 a, requires that console 100-2 be physically modified insome way, or at the very least its exterior casing be opened to exposethis connection point. However, as will also be understood, making thisconnection is possible and does have value as previously describedherein. While a given manufacturer (e.g., Daktronics) typically has morethan one physical console design 100-2, even if each design implements adifferent external keyboard 100-2 k layout and/or a different internalconnector pin-count for attaching to processor 100-2 p, the basicelectronic functionality is the same. Hence, most keyboards work toshort a pair of row/column wires overlaying each other underneath agiven external key and then most keyboards will connect to their consoleprocessor 100-2 p using a ribbon cable. Thus, the present inventionprovides the opportunity to create a juncture A connector that maysupport a variable number of wires depending upon the possible outputsfor a given console's 100-2 keyboard 100-2 k.

Now referring to FIG. 6 a and FIG. 6 b, while juncture A, 100-6A, willprovide the herein taught benefits, the present inventors prefer to workwith juncture B, 100-6B. Using juncture B provides the immediateadvantage of a simple-to-access external connection point that typicallyemploys a universal connector such as a 0.6 cm or ¼″ “tip and ring”(also called a “phono-plug”) connector. This tip and ring connector iscommon to the telephone industry and supports the current-loop signalformat used by the typical manufacturer's console 100-2 to drive theassociated scoreboard 100-3. Thus, it is relatively simple to interfacewith console processor 100-2 p juncture B for the purposes of recordingthe output signals in the training mode as well as scoreboard 100-3juncture B for the purposes of outputting recorded signals in the livemode.

Referring now specifically to FIG. 6 b, rather than simply recordingoutput signals from juncture B, it is possible to by-pass thetraining/recording phase all together based upon a deeper understandingof the digital data embedded in the analog current- loop signal outputby the console processor 100-2 p to the scoreboard 100-3. As will beknown by those familiar with scoreboard technology in general, console100-2 embeds an additional digital signal that represents the variousscoreboard cell segment values onto the “carrier” analog current loopsignal. Hence, the scoreboard 100-3 itself is simply made up of variousdisplay cells, where each display cell can form at least the numeralcharacters of “0” through “9.” By aligning several of these cellssequentially, the scoreboard face represents different game information.For instance, the game clock requires 8 sequential cells:“T,T,M,M,S,S,t,t.” Each individual cell in turn is typically representedas a set of segment values, where the combination of segment valuescreates the various characters—all of which is well known in the art ofelectronic displays and depicted in the lower right hand corner of FIG.6 b as the 100-4-cell (with individually addressable cell segments a, b,c, d, e, f and g).

Hence, the signals transmitted from console processor 100-2 p viajuncture B, 100-6B, to scoreboard 100-3 are actually digital commands toturn on and off individual 100-4-cell segments (a, b, c, d, e, f and g)on the scoreboard face. The transmitted signals are not game commands(such as “start clock”). Thus, as will be understood by a carefulreading of the present teachings, using juncture B, 100-6B, as opposedto A, 100-6A, provides an additional opportunity to eliminate thenecessity of a training mode. In the preferred implementation, thescorekeeper console 100-4 c and 100-4 d simply maintains an internalvirtual scoreboard 100-4-vsb of all of the current individual 100-4-cellvalues to be displayed on the scoreboard 100-3. On a periodic cycle, aCell Display Program 100-4-cdp reads all the cell values on the virtualscoreboard 100-4-vsb and translates their current values intocorresponding digital signals representing the various segments (a, b,c, d, e, f and g) necessary to create their current character (e.g., “0”through “9”). These cell segment digital values are then overlaid bythe. universal scoreboard interface 100-6 onto the carrier analogcurrent loop signal for output to the scoreboard 100-3.

Note that if the Cell Display Program 100-4-cdp is running on thescorekeeper console 100-4 c or 100-4 d (which is preferred,) then theuniversal scoreboard interface 100-6 is simply acting to embed thedigital signals representing the current set of cell segments into theanalog current loop signal being output to scoreboard 100-3. As will beunderstood, it is also possible that the Cell Display Program 100-4-cdpis also run directly on universal interface 100-6, in which casescorekeeper console 100-4 c or 100-4 d must continually provide tointerface 100-6 data representing all current cell values on virtualscoreboard 100-4-vsb to be displayed on the real scoreboard 100-3.

Referring next to FIG. 6 c, there is shown a block diagram of thepreferred universal scoreboard interface 100-6 that is capable ofcommunicating with both a third-party scorekeeping device 100-4 c, 100-4d, such as manufactured by InThePlay, Inc. of Lansdale, Pa. orPointstreak Sports Technologies, Inc. of Thornhill, Ontario, and ascoreboard 100-3 manufactured by a company other than the third party,such as Daktronics, Inc. of Brookings, SD or Fair-play Scoreboards ofDes Moines, Iowa. The purpose of the third-party device 100-4 c, 100-4 dis to interact with an operator for gathering game information thatideally is useful for updating the scoreboard 100-3. The purpose of theuniversal scoreboard interface is to accept the useful game informationas the state of a virtual scoreboard, which it then translates into thenecessary electrical signals for outputting to the scoreboard 100-3,without requiring the use of the scoreboard console 100-2 (as suppliedby the scoreboard 100-3's manufacturer for normal control of theirscoreboard 100-3).

As will be understood by those familiar with the purposes and uses ofthird-party scorekeeping devices, they typically provide more convenientand/or broader functions than the typical scoreboard manufacture'sconsole 100-2, such as but not limited to also interacting with aweb-site and/or a game video recording system. However, while thethird-party devices 100-4 c, 100-4 d have additional benefits, theirdrawback is the inability to control the scoreboard 100-3. Hence, it isdesirable for the manufacturers of the third-party devices 100-4 c,100-4 d to be able to control scoreboards 100-3 from multiple if not allmanufacturers without requiring additional changes to their device. Thepresent invention allows the third-party device to create a virtual dataset in computer memory representing what should be the current state ofthe scoreboard 100-3, which is then translated via the universalscoreboard interface 100-6 into control signals that are recognizable tothe scoreboard 100-3—as if they had been generated by the scoreboardconsole 100-2.

Still referring to FIG. 6 c, third-party scorekeeping device 100-4 c,100-4 d typically includes a physical enclosure containing someprocessing element such as a computer which is then running at least aconsole scorekeeping program 100-4 c-s. While the exact configuration,actions and purposes of the scorekeeping program 100-4 c-s will varybetween third parties, in general the present invention assumes that atleast some of their functions accept game information useful for settingthe current state of the scoreboard 100-3. Typical examples ofscoreboard relevant game information would be game segment (e.g.,“1^(st) period”), team scores and the state of the game clock, if thesport itself uses a game clock. As will be understood by those skilledin the art, the exact nature of the game information should not beconstrued as a limitation of the present invention. As will be furtherunderstood, the present invention is useful even beyond sports, anywherea third-party control device is desirous of taking over control of atypically manually operated information display device, especially butnot limited to devices that employ LED display segment technology.

Through a careful study of the present teachings along with the currentstate of the art in sports scorekeeping and game tracking systems,especially those taught by the present inventors, the reader willunderstand that the third-party scorekeeping device 100-4 c, 100-4 ditself does not need to be manually operated for all of its functions.The device 100-4 c, 100-4 d may in fact receive game information fromrelated systems or devices. For example, device 100-4; 100-4 d mayreceive game information from a performance differentiator such as 200-4(first shown in FIG. 2 and taught by the present inventors in the priorSARTRIA patent). While the exact nature, construction, purposes andtypes of game information created by a performance differentiator suchas 200-4 are in general irrelevant to this aspect of the presentinformation pertinent for display on the scoreboard 100-3. What is mostimportant to understand is that the present invention anticipates thatthe third-party scorekeeping device 100-4 c, 100-4 d may either be apart of a larger system, or be capable of receiving information fromadditional game information devices.

While the present invention also teaches that the universal scoreboardinterface 100-6 can have its own network port 100-6 mb-ip, for thepurposes of either connecting via an alternate path to the third-partyscorekeeping device 100-4 c, 100-4 d (i.e., as opposed to USB ports) orfor connecting to other devices providing game information, it ispreferable that the device 100-4 c, 100-4 d act as the sole aggregatorof game information. In this way, device 100-4 c, 100-4 d can bestmaintain an internal virtual scoreboard representing the exact displayvalues to be output on the scoreboard 100-3, thus limiting the universalscoreboard interface's 100-6 responsibility to translating the virtualscoreboard face into the appropriate and necessary physical controlsignals that will be recognizable to the scoreboard 100-3 (all of whichwas prior taught in relation to FIG. 6 a and especially FIG. 6 b).

Along this same line, a careful reading of the present invention willshow that the relevant game information may also come from alternatedevices, such as a wireless clicker 100-7 (first shown in FIG. 3 b),that are ideally in direct communication with the third-partyscorekeeping device 100-4 c, 100-4 d, i.e., as opposed to directcommunication with interface 100-6; although the alternate is alsopossible without departing from the teachings herein. And finally, ifthe scoreboard 100-3 has a game clock, the third-party device 100-4 c,100-4 d ideally includes either physical buttons . such as 100-4 c-b,e.g., indicating “start clock,” “stop clock,” or “horn sound,” or theirsoftware equivalents. The present invention teaches that these buttonsare physically located in the same enclosure as the third-partyscorekeeping device 100-4 c, 100-4 d, for the convenience of theoperator, but then are internally connected directly to the button portsof universal scoreboard interface 100-6, for reasons to be shortlyexplained. However, those skilled in the art will understand that thesignals from these buttons 100-4 c-b, whether physical or software,could be received directly into console scorekeeping program 100-4 c-sfor interpretation and transmission to interface 100-6, withoutdeparting from the teachings herein.

Still referring to FIG. 6 c, the advantage of having at least one buttonbe both physical and not connected to the universal interface 100-6 viathe scorekeeping program 100-4 c-s, is that in the case of a “consolereset button,” its independence from the device 100-4 c, 100-4 d isextremely useful for implementing what is known in the art as a“watch-dog” program. Before explaining this watch-dog functionality, itis first noted that the third-party device 100-4 c, 100-4-d preferablyreceives its power directly from the interface 100-6, which in turn hasits own console power supply 100-6 mb-ps. Interface 100-6 also includesconsole watchdog program 100-6 mb-w that is capable of communicatingdirectly with console scorekeeping program 100-4 c-s and consolemonitoring program 100-4 c-m via communication paths such as but notlimited to a typical USB port.

Those familiar with software and systems will understand that ascomplexity builds within a device, on occasion it may fail or becomeunstable, which is also assumed to be the case with the third-partyscorekeeping device 100-4 c, 100-4 d. In this situation, since thescoreboard 100-3 is a mission-critical device for a sporting event, thepreferred solution provides for a fast and effective means for resettingthe device 100-4 c, 100-4 d and therefore also its scorekeeping program100-4 c-s. To accomplish this goal, the present inventors prefer havinga physical “console reset button” connected via a physical communicationpath to console watchdog program 100-6 mb-w (as shown in the group ofbuttons 100-4 c-b) that when pressed by an operator causes the followingfunctionality.

First, once the watchdog program 100-6 mb-w determines that the resetbutton has been pressed, then it preferably attempts a successfulcommunication with either or both the console monitoring program 100-4c-m or the scorekeeping program 100-4 c-s. If this communication issuccessful and confirms that the scorekeeping program 100-4 c-s is infact functioning properly, then the watchdog program 100-6 mb-w ideallycommunicates via the third-party scorekeeping device 100-4 c, 100-4 d toinform the operator that no problems are detected and to ask forconfirmation before forcing a reset. If the operator then wishes toproceed with the reset, the watchdog program 100-6 mb-w completes a“soft-reset” by requesting that the scorekeeping device's 100-4 c, 100-4d operating system, or its console monitoring program 1004 c-m forceclosure of the currently running scorekeeping program 100-4 c-s and thenrestart a new copy of the same program. (Note that by using the consolemonitoring program 100-4 c-m, more information can be gathered by thewatchdog program 100-6 mb-w since the console monitoring program 100-4c-m can be programmed with more specific functionality that the genericoperating system, but in either case the present invention should not belimited.)

Still referring to FIG. 6 c and the watchdog functionality, if thewatchdog program 100-6 mb-w determines that the scorekeeping program100-4 c-s is in fact running without a known problem, but either failsto hear back from the operator regarding its request for resetconfirmation, or receives additional reset button signals via group100-4 c-b while waiting for confirmation, then the watchdog program100-6 mb-w proceeds to the “soft-reset” request to close the currentscorekeeping program and then restart a new copy of the same program. Ineither case, whether the reset is confirmed or not, if the reset isconducted, once the new instance of the console scorekeeping program100-4 c-s is up and running properly, it preferably sends a message tothe watchdog program 100-6 mb-w that the “soft-reset” is completed. Ifafter a sufficient time period the watchdog program 100-6 mb-w has notreceived the message that the “soft-reset” is completed, or upon receiptof additional “reset console” signals from button group 100-4 c-b, thenthe watchdog program 100-6 mb-w preferably cuts power to third-partydevice 100-4 c, 100-4 d, thus performing what is known in the art as a“hard-reset.” It is also noted that, when the watchdog program 100-6mb-w first attempts to communicate with the scorekeeping program 100-4c-s to determine if it is in fact “frozen,” unstable, crashed, etc.,then if it does not receive a confirmation signal back from scorekeepingprogram 100-4 c-s, this will also directly trigger the “hard-reset”sequence.

In the “hard-reset” sequence, after cutting and-then restoring power tothe third-party. scorekeeping device 100-4 c, 100-4 d, it is typicalthat device 100-4 c, 100-4 d's onboard computer will be automaticallyrestarted and ultimately end up in a properly running state thatincludes a properly running scorekeeping program 100-4 c-s. Thepreferable scorekeeping program 100-4 c-s also continually retains atleast the current virtual scoreboard state so that if a reset isconducted, upon restarting the scorekeeping program 100-4 c-s can firstcheck for a retained scoreboard state. If a retained state is found,this indicates that a current scorekeeping session was aborted and ifnot found, this means that a normal startup is in effect. If the abortsituation is determined to be the case because a prior scoreboard statewas found, then the scorekeeping program 100-4 c-s preferablycommunicates with the operator to determine if the prior scorekeepingsession (i.e., state of the virtual scoreboard) should be maintained, orif the virtual scoreboard (and therefore also the corresponding realscoreboard 100-3) should now be reset. After a hard reset isaccomplished and the operator presumably confirms that the scorekeepingprogram 100-4 c-s should continue with the last known state of thevirtual scoreboard, then, and in either case, the watchdog program 100-6mb-w's functions are now completed.

As will be understood by those skilled in the art, several variations ofthe watch dog functionality are possible without departing from theteachings herein. Furthermore, the present invention should neither belimited by the type and means of watch dog functionality it implements,nor should it be limited only to scoreboard interfaces 100-6 thatimplement a watchdog program 100-6 mb-w. There are many novel, usefuland beneficial functions described herein with respect to the entirepresent invention as well as the universal scoreboard interface 100-6that are not limited by the lack of a . watchdog program 100-6 mb-w.

Still referring to FIG. 6 c, the universal scoreboard interface 100-6also comprises the scoreboard state program 100-6 mb-p that is capableof receiving virtual scoreboard state changes either and preferablydirectly from the third-party scorekeeping device's scorekeeping program100-4 c-s such as through a USB or similar port (shown but notnumbered), or from some other device via its network port 100-6 mb-ip.(As prior stated, it is also possible that the third-party device 100-4c, 100-4 d communicates its information to the universal scoreboardinterface 100-6 via a network port such as 100-6 mb-ip as opposed tosome other connection such as the USB ports shown.) In any case, what isimportant is that the third-party device 100-4 c, 100-4 d (and any ofits associated systems or devices) provide information that can beinterpreted as indicating the state, or changes to the state, of aphysical scoreboard 100-3. Once the scoreboard state program 100-6 mb-preceives such virtual scoreboard state information, it then proceeds totranslate this virtual information representative of whole scoreboardvalues, such as “2” into the equivalent LED segment values, all of whichwas prior taught in relation to FIG. 6 b. It is herein noted that thisfirst translation step is generally applicable to all types ofcommercially existing scoreboards, where the marketplace has beenreliant upon LED outputs as opposed to video screens.

Once this first translation from whole numeric values into LED segmentvalues has been completed, then the scoreboard state program 100-6 mb-ptransmits any necessary changes to the LED segment values to itsuniversal adapter daughterboard 100-6 db, for instance through a serialport (shown but not numbered). Although many other options are possibleas will be well understood by those familiar with the art, the presentinventors prefer executing the functionality contained withindaughterboard 100-6 db using an FPGA, e.g., as opposed to amicroprocessor or DSP. What is important is that a second translationstep is then performed by scoreboard signal encoder program 100-6 db-eto prepare a proper set of electrical signals containing the informationof the first step translated LED segments for transmittal to thescoreboard 100-3 via the output switch 100-6 db-s. As will be known fromthe teaching herein, these transmitted electrical signals causescoreboard 100-3 to update without the need of an operator activatingthe manufacturer's console 100-2, thus saving a data entry step.

Still referring to FIG. 6 c, it should also be understood that theseparation of the first translation of the virtual scoreboardinformation determined by the third-party scorekeeping device 100-4 c,100-4 d into LED segment values, followed by the second translation intoelectrical signals, does not have to be performed on a separatemotherboard in connection with a daughterboard. However, the presentinventors prefer this implementation since several of the existingscoreboards 100-3 from various competing manufacturers use differentelectrical signals and/or signal connectors. By separating the secondtranslation of LED segment values into proper electrical signals, thepresent invention's design leads to a motherboard 100-6 mb that isgenerally applicable and soft-upgradable to work with any scoreboard100-3, regardless of that scoreboard's physical signal types orconnectors. However, the separation or not of translation functionsshould not be construed as a limitation, but rather as anexemplification thereof.

And finally, still referring to FIG. 6 c, daughter board 100-6 dbpreferably includes a custom port for receiving signals from themanufacturer's console 100-2 for direct pass-through via switch 100-6db-s to scoreboard 100-3. As will be understood by those familiar withscoreboards and their environment, this allows the scoreboard to beoperated “as-is” in those situations where the third-party device 100-4c, 100-4 d is not applicable, available or desirable. While not to beconstrued as a limitation, the preferred scoreboard state program 100-6mb-p automatically detects if a third-party scorekeeping device 100-4 c,100-4 d is trying to control scoreboard 100-3, in which case it eitherasks for operator confirmation (using scorekeeping device's 100-4 c,100-4 d own interface), or it immediately allows such direct control totake place by signaling to encoder 100-6 db-e to set switch 100-6 db-sto connect its own encoder output; as opposed to manufacturer's console100-2 output, to scoreboard 100-3.

Referring next to FIG. 7 a, there is shown session processor 200-5 thatwas first taught by the present inventors in the prior SARTRIAapplication. The purpose of session processor 200-5 is to accept andtranslate into a session index one or more human or machine observations100-4 m, 100-5 m, respectively, being made about a performanceconcurrent with the recording of that performance (e.g., video and/oraudio). As prior taught, the session processor 200-5 performs itstranslation functions under the direction of external rules 200-5 r,thus making it externally programmable and therefore also moreuniversally adaptable to various types of performances (“sessions”),such as but not limited to sporting, theatre and music events.Furthermore, all human and machine observations 100-4 m, 100-5 m,respectively, are packaged into a common “mark” 200-5 c-m dataset thatminimally defines the type of mark (e.g., “goal mark”), the session timeof mark and any related data (e.g., “player, assist 1, assist 2, etc.”).As the marks 200-5 c-m are received by the session processor 200-5 andprocessed according to the external rules 200-5 r, they may serve tocreate, start or stop one or more associated types of events 200-5 c-e(where lower case “e” event refers to a limited duration activitydetected within the performance, e.g., “scoring a goal,” vs. the entireperformance itself, which is commonly referred to uppercase “E” Event).It is the combination of these limited duration activities, or events200-5 c-e, that together form a session index into the recording(s) ofthe performance—all as previously taught.

Still referring to FIG. 7 a, what is new is that the session processor200-5 now includes an additional AES output filter 200-5 f forcontrollably selecting and packaging a real-time stream of event 200-5c-e changes with related data and observation marks 200-5 c-m withrelated data to automatic entertainment system (AES) session contentdata based 400-i. As will be understood by a careful reading of thepresent inventors' prior SARTRIA application, session processor 200-5may potentially receive a significant number and range of observationmarks 100-4 m and 200-4 m. For instance, for sporting Events the humanobservations 100-4 m may come from the game officials 100-1 a, thescorekeeper through console 100-4 a, 100-4 b, 100-4 c or 100-4 d; or aremote observer through observation entry device 100-2 a. Also at asporting Event, while the present inventors have taught and prefer theuse of machine vision object tracking systems such as 200-3, othertracking systems based upon RF, IR, UWB, GPS, etc. already exist and areincorporated within the present and prior teachings.

As will be clear to those familiar with the preferred and anticipatedentertainment devices 400-J5, 400-J6, 400-J7, 400-J8 a, 400-J8 b and400-J9 first listed with respect to FIG. 5, not all of the sessioncontent 200-5 c (including marks 200-5 c-m and events 200-5 c-e) isimportant for controlling their functions. Thus, while not necessary,the present inventors prefer adding AES output filter 200-5 f to sessionprocessor 200-5 for the purposes of creating limited AES session content400-i under the control of external rules 200-5 fr. As will beunderstood by those skilled in the art of software systems, manyvariations are possible for providing filter 200-5 f, with or withoutthe preferred external rules 200-5 fr, outputting AES session content400-i in various possible formats. What is most important is the datacollected in real-time about the current performance are provided to oneor more entertainment devices such as but not limited to 400-J5, 400-J6,400-J7, 400-J8 a, 400-J8 b and 400-J9. While it is preferred, the datacollected do not need to be represented as observation marks 200-5 c-mor event 200-5 c-e changes, nor do they need to come through a sessionprocessor 200-5, but rather the data collected could come directly fromthe observing devices, e.g., the scorekeeper console 100-4 a, 100-4 b,100-4 c, 100-4 d or the object tracking system 200-3; many variationsare possible and anticipated without departing from the scope of theteachings herein provided.

Referring next to FIG. 7 b, there is shown the preferred automaticentertainment processor 400-1 comprising first a UAC command issuer400-1 a and second one or more device wrappers 400-1 b. Command issuer400-1 a preferably receives a filtered real-time stream of AES sessioncontent 400-i for triggering the issuance of one or more universalcommands 400-1 uac to be sent to one or more entertainment devices400-J5, 400-J6, 400-J7, 400-J8 a, 400-J8 b and 400-J9, thus effectingtheir actions (i.e., state changes 400-1 ds). As prior discussed inrelation to FIG. 7 a, AES session content 400-i preferably includesreal-time observations being made about the ongoing performance byeither operators using input devices referred to as human observations100-4 m or automatic machines making machine observations 200-4 m.Regardless of how the observations are made, regardless of how they areprovided (e.g., via a session processor 200-5,) and whatever theirassociated data formats (e.g., marks 200-5 c-m and events 200-5 c-e),this AES session content 400-i serves as a trigger for turning on andoff various features (actions) of any one or more entertainment devices400-35, 400-36, 400-J7, 400-J8 a, 400-38 b and 400-J9.

While not necessary, the preferred automatic entertainment processor400-1 operates under the direction of a distinct set of external triggerrules 400-1 tr for each distinct type of entertainment device 400-J5,400-J6, 400-J7, 400-J8 a, 400-J8 b and 400-J9 to be controlled. As willbe understood by those skilled in the art of software systems, ratherthan using generalized external rules so that the entertainmentprocessor 400-1 becomes externally programmable, it is feasible tohard-code the decision logic herein described directly in processor400-1. Thus, while hard-coded software is not preferred, it will stillperform the primary function taught herein of affecting the real-time,automatic control of one or more entertainment devices such as 400-J5,400-J6, 400-J7, 400-J8 a, 400-J8 b and 400-J9, based upon informationoriginated by either humans and/or machines regarding the ongoingperformance. As will also be understood by those skilled in the art ofsoftware systems, there are many ways of creating external rule sets400-1 tr that can be processed in combination with the AES sessioncontent 400-i in order to automatically select (or not) from one or morepotential universal commands 400-1 uac. The present inventors willdefine their preferred implementation of UAC command issuer 400-1 a withrespect to upcoming FIGS. 8 a, 8 b, 8 c, and 8 d; suffice it to say thatthe pattern of implementation is similar to that adopted within theprior application disclosing the session processor's 200-5 functions.

Referring still to FIG. 7 b, for each unique device type such as 400-J5,400-J6, 400-J7, 400-J8 a, 400-J8 b and 400-J9, UAC command issuer 400-1a ideally maintains a device state 400-1 ds table that itself is used asadditional input along with external trigger rules 400-1 tr and AESsession content 400-i for determining subsequent issuance of specificuniversal action commands 400-1 uac. For example, in the case where thedevice type is an audio system 400-J5, the relevant AES session content400-i would at least include the observation marks 200-5 c-m of:“clocked stopped” and “clock started.” Realistically, they should alsoinclude the “game started,” “period started,” “period stopped” and “gamestopped” marks. Assuming that the startup device state 400-1 ds of themusic system 400-J5 is “off,” then external rules 400-1 tr will indicateto the UAC command issuer 400-1 a that: “IF game started AND periodstarted AND clock stopped” THEN issue universal action command 400-1 uac“Start Music.” Conversely, “IF device state=on AND (clock started ORperiod stopped OR game stopped)” THEN issue universal action command400-1 uac “Stop Music.” While this present example is representative inthat it includes the three preferred types of control information inputto AES processor 400-1 a as well as the two types of action informationoutput by UAC command issuer 400-1 a, it should not be in any wayconstrued as limiting. Hence, there are many well-known higher-levellanguage constructs that are equally capable of expressing the necessarylogic.

What is most important to the preferred teachings is the three inputdata sets including AES session content 400-i, external trigger rules400-1 tr and device state 400-1 ds along with two output data setsincluding device state 400-1 ds and universal action commands 400-1 uac.As prior stated, while the AES processor 400-1 a could be hard-coded andthus not require external trigger rules 400-1 tr, to accomplish theherein taught minimal functionality, it must receive/know sessioncontent 400-i in some format indicative of the current state of theperformance as well as the current device state 400-1 ds of theentertainment device (e.g., 400-J5) being controlled. As will be wellunderstood by those familiar with software systems, it is not necessarythat UAC command issuer 400-1 a issue a universal action command 400-1ds that itself must then be “locally interpreted” by each unique device(e.g., a Kintronics Information Technology “PA over IP” system) of agiven device type (e.g., 400-J5 PA and Music System), since commandissuer 400-1 a could just as well have issued the exact device specificcommands directly interpretable by the given unique device. (As is wellknown in the art, these “device specific commands” are typicallypre-established in a software application interface (API) or equivalentthat is usable for electronically and programmatically controlling thegiven device.)

Thus, still referring to FIG. 7 b, while not necessary UAC commandissuer 400-1 a ultimately and preferably outputs one or more universalaction commands 400-1 uac in real-time across any necessarycommunication path usable for electronically controlling a given uniquedevice (e.g., a Kintronics Information Technology “PA over IP” system).As will be well understood by those familiar with software systems,using the preferred approach of issuing universal commands 400-1 uac(e.g., “start music” or “stop music”) has many advantages. First, thisapproach supports the best practice of loose coupling between the UACcommand issuer 400-1 a and the specific device wrapper 400-1 b (e.g., aKintronics Information Technology “PA over IP” system), which in turnmeans that the external rules 400-1 tr may be pre-established for thegiven device type (e.g., 400-J5 PA and Music System) in general, priorto actually selecting the exact device (and therefore wrapper 400-1 b)to be implemented (again for example a Kintronics Information Technology“PA over IP” system). Another obvious benefit is the reduction inhard-coded logic within UAC command issuer 400-1 a that tends over timeto become outdated and difficult to maintain, all of which is familiarto those skilled in the art of software systems.

As will also be well understood by those familiar with software systemsin general and the programming of devices with software API's inparticular, each universal command 400-1 uac that is intended to effectthe actions of a given device type (e.g., 400-J5 PA and Music System)must then be translated into one or more actual API commands recognizedby the specific device (e.g., a Kintronics Information Technology “PAover IP” system). As shown in FIG. 7 b, the present inventors prefer theuse of an AES device wrapper 400-1 b software construct that includesembedded methods already capable of receiving any universal command andthen invoking an associated method for translating that command into thecustom API sequence(s) (e.g., 400-J5-API) necessary to accomplish thedesired resulting device action. While various software implementationsof the AES device wrapper 400-1 b are possible, the present inventorsprefer that the specific custom API sequences (e.g., a sub-set of400-J5-API) to affect the behavior of a specific device (e.g., aKintronics Information Technology “PA over IP” system) for a specificuniversal action command (e.g., “start music”) be stored as dataexternal to the device wrapper 400-1 b itself. While this is preferredit is not necessary for the present invention and should not beconstrued as a limitation. The present invention would still perform itsbasic intended function if the translations of universal commands intocustom API sequences were hard-coded within wrapper 400-1 b. As priormentioned and as will be well understood by those familiar with softwaresystems, this technique for externalizing such data tends to supportmore robust and maintainable software.

Still referring to FIG. 7 b, after human and/or machine observations100-4 m, 200-4 m (as contained in the real-time stream of AES sessioncontent 400 i) are converted into a real-time stream of universal actioncommands 400-1 uac by UAC command issuer 400-1 a using external triggerrules 400-1 tr and knowledge of each entertainment device's state 400-1ds, the commands 400-1 uac are then converted into a real-time stream ofcustom API sequences (such as 400-J5-API through 400-J9-API) by AESdevice wrappers 400-1 b, such that a corresponding real-time stream ofcustom action commands is provided to one or more specific entertainmentdevices (of types such as 400-J5, 400-J6, 400-J7, 400-J8 a, 400-J8 b and400-J9) for creating various entertainment, information or otherwiseperformance effects.

Referring next to FIG. 7 c, there is shown a table of the preferred datafields comprising a universal action command 400-1 uac. Those skilled inthe art of software systems and in particular databases will be familiarwith the UAC Data Fields as described in the table and without furtherexplanation needed (where the initials “Opt” to the left of the UAC DataField column means “optional”). As will also be understood, while theportrayed UAC Data Fields are preferred, they are not necessary as othervariations are possible without departing from the scope and teachingsof the present invention. What is important with respect to thepreferred universal action command 400-1 uac is that it holds sufficientinformation to be processed by multiple unique entertainment devices(e.g., as sold by different manufactures such as Kintronics, Barix orStentofon) that each represent the same single type (e.g., 400-J5 PA andMusic System). The present inventors further prefer that the universalaction command format be sufficient for processing by all uniqueentertainment devices of all types (such as 400-J5, 400-J6, 400-J7,400-J8 a, 400-J8 b and 400-J9). As will be understood by those familiarwith software systems and databases, beyond the specified UAC DataFields and their purposes as depicted, the exact formats andrepresentation of the proposed fields are also immaterial to the presentinvention.

Referring next to FIG. 7 d, there is shown a table of exemplaryentertainment devices including 400-J5, 400-J6, 400-J7, 400-J8 a, 400-J8b and 400-J9 and others. Along with each example of the device type,there is also listed its general purpose, data source type and use aswell as example data outputs. Those familiar with the device typesherein discussed and reviewed in FIG. 7 d will understand theentertainment value provided by the listed example data outputs. Thoseskilled in the art of software control systems will understand that thepresent invention sufficiently teaches apparatus and methods forautomatically controlling the listed device types to accomplish theirexample data outputs, all in coordination with the on-going progress ofa performance. Hence, the reader will understand that the presentinvention provides significant value by at least allowing a performanceto be enhanced without additional labor costs for the operation of thelisted entertainment devices, or any similar devices, such as listed inFIG. 7 d as “other” under the heading “Part.”

Still referring to FIG. 7 d and also to FIG. 7 c, special attention isdrawn to the following teachings. First it is noted that the data fileto be output (or transmitted) to a given entertainment device may beintended either for direct expression through the device (e.g., text tobe converted to speech or shown on a display, or a video to be shown ona display), or it may be a script file specifying control attributes tobe varied on the entertainment device (e.g., brightness, color, foglevel, volume). Second, the source of the data may either be attached tothe UAC 400-1 uac itself, it may be fetched from a database, or it maybe from an ongoing data stream. The database or stream could be eitherlocal or remote to the automatic entertainment system 400. Oneanticipated use of an external database is to hold a list of music forimmediate and automatic purchase and download by system 400. Thisdatabase might be maintained by some third party and made generallyavailable via the internet. It is further anticipated that the databasewould include songs that have been tagged with several key fields, or atleast semantic tokens for their variable selection. One ideal and wellunderstood storage methodology for accomplishing'this type offunctionality would be to use a SQL server database with multiple searchfields for each song therein contained, such as but not limited to andfor example with respect to the sport of ice hockey:

-   -   Session Type, e.g., “Sport, Ice Hockey, Game”;    -   Competition Level, e.g., “High School”;    -   Performance event, e.g., “Play Stopped, Home Goal, Away Goal,        Home Penalty, Away Penalty”;    -   Score Differential, e.g., “H0, H1, H2, H3, H4, etc.” meaning        that the Home Team is ahead by 0, 1, 2, 3, 4, etc. goals, with        similar values for the Away Team;    -   Time of Game, e.g., “P1:00, P1:05, P1:10, P1:15, P1:99, P2:00,        etc.” meaning the period in five minute increments where “99”        means the last minute in the period; and/or    -   Theme, e.g., “Sporty, Hard Rock, Popular, Science Fiction, etc.”

As will be understood by those familiar with sports in general, thesetypes of information are either readily available from the officialscorekeeping system 100, 101, 102 or 103, or are preferences (e.g.,“Theme”) that can be easily pre-set prior to the performance. Hence,when each UAC command 400-1 uac is generated for retrieving a data filefrom a database (such as a song from a remote third-party database), theUAC 400-1 uac preferably includes an search string (such as an SQLselect statement) for determining which data files stored in thedatabase should be included in the possible list of choices for output(where the SQL select statement is referred to in FIG. 7 c as the DataSource Inclusion Filter). Using this teaching, the third-party musicsupplier could maintain a list of songs being updated over time that aretagged for appropriateness to a given sport, age level, situation andpreference, etc. The automatic entertainment processor 400-1 is thenresponsible for selecting one or more potential songs, after which itmay then randomly select one song, or simply take the next song on thelist with the possible exclusion of any songs already taken for the sameset of inclusion filter parameters, if two or more potential songs arereturned from the queried database—all as will be understood by thosefamiliar with controlling the play of music at a sporting event. Thepresent inventors also anticipate that the “included” list of potentialsongs retrieved by the SQL select statement (or similar database searchtechnique) could have a second filter or multiple exclusion filtersapplied prior to the final selection. An example exclusion filter forsongs would be a specific list of individual songs that thoseresponsible for the performance do not want output, even though thesongs (data files) otherwise match the normal search criteria—thusproviding the local entertainment system 400 user with a way ofoverriding the choices made possible by the third-party databaseprovider.

While not depicted in the figures herein, it will be known to thoseskilled in software systems in general and internet ordering systems inparticular, that the third-party database supplier could track the usageof their data and conduct appropriate automatic billing. The presentinventors specifically include this type of automatically searchabledatabase of output data files, especially including songs and videoclips, as a teaching of the present invention. It is further anticipatedthat the database is ideally stored remotely with internet accessavailable, and that the database is connected to an automatic billingsystem for accounting for the transaction.

The following FIGS. 8 a, 8 b, 8 c, and 8 d are all taken directly fromthe present inventors' prior related application entitled SESSIONAUTOMATED RECORDING TOGETHER WITH RULES BASED INDEXING, ANALYSIS ANDEXPRESSION OF CONTENT, also herein referred to as SARTRIA.

Referring next to FIG. 8 a, amongst other important new teachings, thisprior SARTRIA application defined a universal protocol for normalizinghuman and machine “external observations” into “marks” (a step called“differentiation”) as well as a “session processor” for both“integrating” these “marks” into “events” and then “synthesizing” thecombined mark and event “session knowledge” into “summary and tertiarymarks” (or “internal observations”). The session processor also includeda final stage referred to as “expression” where, for instance, theevents could be associated into custom “foldering trees” for laterorganized retrieval—thus serving as a “session index” into all recorded“session content” such as video and audio.

Still referring to FIG. 8 a (and to FIG. 5), while the present systemtaught herein prefers the use of a content generation system 201, andeven more specifically the object tracking, performance differentiator,session processor solution prior taught, it will be known to thoseskilled in the art that other software implementations are possiblewithout departing from the present teachings for interfacing with andcontrolling an automatic entertainment system 400. Furthermore, thepresent inventors have depicted the automatic entertainment processor400-1 as a subsequent service to be performed after the initial work ofthe content generation system 201 in general, and the session processor200-5 in particular, has created session content 200-5 c; where thecommunication between the two is primarily through AES session content400-i (a derivative of session content 200-5 c implemented as areal-time data stream). Those skilled in the art of software systemswill understand that sufficient uses and specifications for a separatepreferred automatic entertainment processor 400-1 have been hereintaught; where these teachings in summary include programmable controlvia external trigger rules 400-1 tr, the monitoring of entertainmentdevice states 400-1 ds, the issuance of universal action command 400-1uac and then finally the translation of the universal command 400-1 uacinto specific API sequences (such as 400-J5-API) for controlling aspecific entertainment device (see especially FIG. 7 c).

Now exclusively referring to FIG. 8 a, the present inventors show thatthe prior taught session processor (also herein preferred for use incontent generation system 201), with one variation to be discussed inrelation to FIG. 8 d, is further usable and preferred for implementingat least the UAC command issuer 400-1 a portion of the automaticentertainment system processor 400-1, including all of issuer 400-1 a'spreferred features of programmable control via external trigger rules400-1 tr, the monitoring of entertainment device states 400-1 ds and theissuance of universal action commands 400-1 uac. Furthermore, based uponthe teachings of the prior SARTRIA application, two distinctconfigurations are possible. In the first case, as depicted in FIG. 5,the UAC command issuer 400-1 a (as implemented using a session processorsimilar to 200-5) can remain connected to the output of the “sessionindexing” session processor 200-5. Hence, in this first case there wouldbe two separate session processors connected in series. In the secondcase, the two session processors can be connected in parallel. As willbe understood by a careful reading of the prior taught SARTRIA patentapplication, the teaching anticipated these various serial or parallelconfigurations and specifically included a “mark messaging pipe” (e.g.,see SARTRIA FIGS. 32 a, 32 b) for the purposes of carrying any and allhuman or machine observations (such as made through or by “externaldevices” for example scorekeeping system 101, 102, 103 or performancedifferentiator 200-4) to the input port of one or more sessionprocessors working in parallel. As a further teaching, it was shown thatevery session processor is able to connect its output port to the samemark message pipe (e.g., see SARTRIA FIG. 38 b), thus supporting aninfinite ability to nest session processors as needed for the desiredsession content transformations—all as will be well understood by thosefamiliar with software systems in general, and object-orientedprogramming in particular. And finally, as will also be understood, asingle session processor such as herein depicted 200-5, could performboth the processing of session content 200-5 c simultaneous with the UACcommand issuance 400-1 a, thus incorporating the first portion of theautomatic entertainment processor 400-1 into the content generationsystem 201.

Still referring to FIG. 8 a, the reader is directed to the boldedbracketing, numbering and accompanying descriptions that are shown in aneffort to relate the original teaching to that herein specified.Specifically, in the lower right of FIG. 8 a, the original SARTRIAperformance area 1 is herein referred to as 100-1. Just to the left ofperformance area 1, there is shown the original external devices (e.g.,in the present application including but not limited to scorekeepingsystem 101, 102, 103 or object tracking system 200-3). These externaldevices then generate external device data and primary marks (originally3-pm) now 100-4 m, 200-5 m in a stage referred to as differentiation30-2 (herein including performance differentiator 200-4). The presentinventors herein teach that in the prior taught second stage ofintegration 30-3, primary events (originally 4-pe) may now be used torepresent a single entertainment device type, e.g., 400-J5, 400-J6,400-J7, 400-J8 a, 400-J8 b and 400-J9. As will be understood from acareful reading of the prior SARTRIA application, an event type is atemplate object meaning that it serves as pre-knowledge used to createone or more actual instances of the type during a session. In this way,if an event type is used to represent an entertainment device type(e.g., 400-J5 PA and Music System), then during the processing of anongoing session an actual instance of the event type can be used torepresent the current “on/off” state of an individual device (e.g., adistinct Kintronics Information Technology “PA over IP” system).

Hence, when it is understood that the (actual) event instances of agiven (template) event type form a digital waveform over session time,then it will also be understood that this waveform can be used torepresent the transition points (i.e., changing device states 400-1 ds)of a specific (i.e., actual) entertainment device. (For examples of thiswaveform teaching in the SARTRIA application, see FIGS. 26 a, 26 b, 26c, 28 a, 28 b and 28 c). Thus, when a specific device (such as theKintronics system 400-J5) is sent a universal action command 400-1 uacto start outputting a data file, i.e., it is “turned on,” then an actualevent instance can be created and started (thus going “high” inelectronics terminology). Likewise, when the device is sent a command400-1 uac to stop outputting its data file, i.e., it is “turned off,”then the created and started actual event instance can be stopped (thusgoing “low”). As will also be understood, if the single device type(again, e.g., 400-J5) has multiple distinct actual devices (such as twoseparate Kintronics systems), then two event instances could be created,started and stopped in parallel, where each create, start and stop markcarries a unique identifier for the specific Kintronics system as one ofits related datum (all as will be understood in light of the priorSARTRIA teachings).

Referring still to FIG. 8 a, once it is understood that primary events4-pe may be used to represent actual entertainment device states 400-1ds for individual device types (e.g., 400-J5), it will also be seen thatthese current device states (i.e., their representative actual eventinstances), in combination with the incoming marks 3-pm (representinghuman 100-4 m and machine 200-4 m observations), may be acted upon bymark effects events rules 2 r-i—all as taught in the prior SARTRIAapplication. Hence, these session processor mark effects events rules 2r-i are then serving as the herein specified UAC command issuer 400-1a's external trigger rules 400-1 tr, thus allowing issuer 400-1 a toprogrammatically respond to the stream of AES session content 400-i (orpreferably directly to observations 100-4 m, 200-4 m) for theconditional issuance of universal action commands 400-1 uac based uponthe current device's known state 400-1 ds (all as previously specifiedespecially in relation to FIG. 7 b).

And finally still in reference to FIG. 8 a, as will be understood by acareful reading of the prior SARTRIA application, during the synthesisstage 30-4 the session processor is able to automatically make its own“internal observations” represented as secondary and tertiary marks. Forthe purposes of the present invention, it is preferred that tertiarymarks 3-tm are used as universal action commands 400-1 uac. As will bediscussed further in relation to upcoming FIG. 8 c, these tertiary marks3-tm are also controllable via external (calc) rules 2 r-c, which thenare acting as an extension of the herein taught and anticipated externaltrigger rules 400-1 tr. Specifically, if the mark effects events rules 2r-i are used to start a single actual event instance representing allspecific devices (i.e., rather than one actual instance for eachspecific device), then the tertiary rules 2 r-c may be used to issuemultiple tertiary marks 3-tm, one for each specific device. Thus, eachissued tertiary mark 3-tm would carry a related datum properlyaddressing it to a single specific device, which in turn means that thesession processor acting as a UAC command issuer 400-1 a would issue onemark 3-tm to serve as one universal action command 400-1 uac for eachspecific entertainment device (such as multiple Kintronics systems) asdictated by tertiary rules 2 r-c.

Of course, several variations of these teachings of applying a sessionprocessor to implement the preferred UAC command issuer 400-1 a arepossible, at least including that only one tertiary mark 3-tm (i.e.,universal command 400-1 uac) is issued for each event type 4-pe (i.e.,device type such as 400-J5), regardless of the number of specificentertainment devices of the given type (e.g., two or more Kintronicssystems). In this case, all of the two or more Kintronics systems wouldreceive the same universal action command 400- 1 uacand thus react inthe same manner. Even if the two or more specific entertainment devices(or the same general type, e.g., 400-J5 PA and Music System) are fromdifferent manufacturers (e.g., one Kintronics system and one Stentofonsystem), they can still receive the same universal action command 400-1uac(i.e., as mark 3-tm) since their AES device wrapper 400-1 b willproperly translate the universal -command 400-1 uac into appropriatecustom API sequences (e.g. 400-J5-API), all a discussed in detail withrelation to FIG. 7 b.

Thus the careful reader will see that the prior taught steps of theintegration and synthesis of session content will suffice for alsoimplementing the herein taught UAC command issuer 400-1 a as a sessionprocessor similar to 200-5. As will be known to those skilled in the artof software systems, doing this has several advantages, not the least ofwhich is the reusability of existing software objects.

Referring next to FIG. 8 b, there is shown a detailed diagram of themark and event objects and their relationships as taught in the SARTRIAapplication. As can be seen, an event type 4-a is shown to have durationover session time depicted as horizontal length. The event type 4-a isalso shown to have three distinct points of creation, starting andstopping—all as triggered by mark types 3-x, 3-y and 3-z, respectively.Also portrayed are rule stacks 2 r-i governing the connection (oreffect) of a given mark 3-x, 3-y and 3-z on its associated event 4-pe.Also shown are the new uses for this prior teaching such that eventtypes 4-a are useable to represent entertainment device types, e.g.,400-J5, where actual event type instances therefor represent that devicetype's device state 400-1 ds (i.e., “high” means the device is currentlyon, or outputting, while “low” means the device is currently off, or notoutputting). Furthermore, as also taught herein, marks 3-x, 3-y and 3-zfor creating, starting and stopping the entertainment device's outputare equivalent (but not limited to) human observations 100-4 m, such asmade through scorekeeping systems 101, 102, 103, and machineobservations 200-4 m, such as made by performance differentiator 200-4.And finally, rules 2 r-i for determining if a given mark 3-x, 3-y and3-z/observation 100-4 m; 200-4 m should create, start or stop an actualevent 4-a/device type's state 400-1 ds incorporate the intended andpreferred functions of the external trigger rules 400-1 tr.

Referring next to FIG. 8 c, there is shown a combination node diagramwith a corresponding block diagram detailing the relationship betweenthe tertiary mark (Mc) (i.e., the UAC command 400-1 uac) and itspotential triggering objects such as another mark type (i.e.,observations 100-4 m; 200-4 m) or an event type (e.g., device 400-J5state transitions 400-1 ds). As will be understood by a careful readingof the prior SARTRIA patent application, each mark type template may beassociated with a zero or more context datum, where each datum defines aunique piece or set of information that will ultimately become acorresponding related datum when an actual instance of the mark type iscreated from the template. What is first noted by the present inventorsis that the creation and specification of each context datum associatedwith a (tertiary) mark type (i.e., in this implementation universalaction command 400-1 uac) is controlled by a rule stack. As priordiscussed, these rule stacks are sufficiently capable for drawing fromany existing session content 200-5 c and combining or otherwisecalculating to create a new datum. For the purposes of the presentinvention, these rule stacks perform another of the specified andpreferred functions covered under the overall scope of external triggerrules 400-1 tr shown in FIG. 7 b. In this case, the context datum beingdefined by the rule stack would represent the various universal actioncommand 400-1 uac data fields as detailed in FIG. 7 c.

Still referring to FIG. 7 c, as will be understood by those familiarwith both software systems and the teachings of the prior SARTRIAapplication, rule stacks may refer to either variable session contentdefined as the session is in progress, or constants established prior tothe session. As will also be understood, all the preferred UAC datafields listed in FIG. 7 c fit into either of these two data sourcetypes. For example, the following UAC data fields are ideally“pre-known” for which constants may be used for their specification:Device Type, Device ID, Data Source Type, Data Source Connection, DataSource Name, Data Source Use, Data Source Exclusion Filter(s), Data FileFormat, Data File Start/Stop Control, Initiation Transition Script,Termination Transition Script and Command. The remaining UAC data fieldsof Data Source Inclusion Filter and the actual Data File might bepre-known or also generated “on-the-fly” using the current sessioncontent 200-5 c. (The creation of these two variable UAC data fieldswill be discussed in greater detail with respect to FIG. 8 d.)

The present inventors note that the preceding lists of example“pre-known” data fields vs. “on-the-fly” data fields is to be consideredas exemplary rather than limiting. As will be understood by thosefamiliar with software systems and especially the teachings of the priorSARTRIA application, many of the “pre-known” data fields could beimplemented as variable data sources taken for instance from what wasprior taught in the SARTRIA application as the “session registry.” Thepurpose of the session registry was to serve as a list of all external“input” devices that the session processor should specifically be awareof—hence, any of these registered external devices might at some pointbe transmitting valid observations for integration, synthesis andexpression. The present inventors prefer that the external “output”devices, e.g., entertainment devices such as 400-J5, 400-J6, 400-J7,400-J8 a, 400-J8 b and 400-J9 also use the same registry. Furthermore,the present inventors prefer that each local and remote database or datastream also be “registered” as they serve as session information input.Thus the reader will understand that each of the registeredentertainment devices, databases or data streams will have associatedtheir own particular Device ID, Data Source Connection, Data SourceName, Data File Format and Data File Start/Stop Control, all as “relateddatum” or similar variable data. Hence, the context datum rule stackused to create any given UAC data field above listed as “pre-known,” maypreferably use variable data sources to alternately retrieve thispre-known data from the session registry.

As will be understood by those familiar with rule-based softwaresystems, rather than using constant values, it is preferable to havecontext datum specification rules that refer to variable data sourceswhich themselves are pointed to a session registry. In this way, onesingle set of rules can service any number of sessions being performedat potentially different locals with different entertainment devices,databases, etc (i.e., presumably with different session registryentries). While this type of data factoring is a critical teaching ofthe prior SARTRIA application, it is merely being restated here as amatter of showing that the session processor and all of its features areusable for implementing the preferred UAC command issuer 400-1 a.Regarding the UAC data fields being pre-definable as context datumassociated to a tertiary mark (representing a UAC command 400-1 uac), itshould also be obvious that the data fields of Initiation TransitionScript and Termination Transition Script are forms of a database and assuch could likewise be defined as registered devices.

And filially, still referring to FIG. 8 c, for each tertiary mark typeserving as a universal action command 400-1 uac, the prior SARTRIAteaching provided for what was called a “trigger object.” This objectcould either be another mark or an event. For the purposes ofimplementing the UAC command issuer 400-1 a, the preferred triggerobject is of course the event type 4-pe established to represent theentertainment device (e.g., 400-J5) going through its on/off devicestate 400-1 ds changes. Hence, a simple association between the “PA andMusic System” event type and the UAC tertiary command mark representing“start music” is sufficient to trigger the automatic issuance of the UACcommand 400-1 uac. As will be further understood by a careful reading ofthe SARTRIA application with respect to its corresponding FIG. 31, the“set time” parameter associated with the event type trigger can be setto either “event start” or “event stop.” Hence, in this example, the settime would be set to “event start,” which is also carrying the meaningthat the music system's device state is “on” (as previously discussedespecially in relation to FIG. 8 a). In a similar manner, a second UACtertiary command mark is preferably associated with the same “PA andMusic System” event type for issuance of the “stop music” UAC command400-1 uac. In this case, the set time is obviously set to be at “eventstop.”

It is also noted that the original SARTRIA teaching included a rulestack in association with the trigger object, whether the trigger was amark or in this AES example case an event type. This rule stack is meantto provide a conditional test before simply issuing the universalcommand mark to “start music” or “stop music.” However, as will beunderstood by a careful reading, since the event type itself representsthe entertainment device's state, the rule stack which governs thatevent's starting and stopping is sufficient for also triggering the UACcommand 400-1 uacissuance. Therefore the reader will understand that thetrigger object associated with the universal command mark does notrequire an associated rule stack for the preferred implementation of theUAC command issuer 400-1 a.

Referring next to FIG. 8 d, there is shown the original SARTRIA teachingfor a “descriptor.” In general, a descriptor, also called a descriptorstack, is capable of conditionally concatenating any number of tokensinto strings, where an individual token may itself be another string.Furthermore, any individual token could be a constant or a variabledrawn from any existing session content 200-5 c. Tokens could also haveeither, neither or both a pre-fix and suffix. A thorough understandingof the SARTRIA teachings will show that this powerful teaching allowsfor the simple creation of an object name, e.g., “Home Goal 3,” or themore complex creation of a prose description, e.g., “At 15:07 in thefirst period, number #17 Hospodar took a pass from #29 Donavan to putthe Jr. Flyer's up 1 to 0, which was enough for a victory as the Jr.Flyer goalie Aman stopped all 23 of the Colonials' shots.” In the priorSARTRIA application, these descriptors were used to automaticallygenerate names and descriptions for the various events being created,started or stopped.

Still referring to FIG. 8 d, the present inventors teach the additionaluse of descriptors for creating these same conditionally concatenatedstrings for any context datum (e.g., those related to a tertiary markrepresenting a UAC command 400-1 uac,) not just an event type. Thisteaching is represented in the node diagram at the top left dotted boxarea of FIG. 8 d as the association line joining the context datum (CD)with the naming rule (L). For the present example of implementinguniversal action commands 400-1 uac with tertiary marks and theirrelated datum (as defined by a template context datum), the two UAC datafields most benefited by using a descriptor stack are the Data SourceInclusion Filter and the Data File. As was previously discussed hereinwith reference to FIGS. 7 c and 7 d, the Data Source Inclusion Filtercould be an SQL select statement for querying a database of potentialData Files for output by the desired entertainment device, for example adatabase of music for output by a PA and Music System 400-J5. As will beunderstood by those familiar with software databases that implement aSEQUEL (SQL) relational database model, an SQL select statementcomprises a combination of standard keywords and clauses (such as“SELECT,” “FROM,” “WHERE,” “GROUP BY,” etc.) along with their associatedvariables. As will also be understood, the descriptor tokens set to aconstant may be used to implement the keywords or clauses while thetokens set to a session content variable may be used to as the SQLclause variables. (Note that it is also possible to use the prefix orsuffix to a variable descriptor token for representing the keyword orclause, i.e., rather than creating an additional constant descriptortoken.)

What is most important to be understood is that the descriptor object asprior taught in the present inventor's SARTRIA application is useableas-is in association with any context datum to adequately create atleast both simple and complex event descriptions as well as SQL selectstatements. The event descriptions (such as “Great save on shot 28 byhome goalie David Aman”) are ideal as automatically generated Data Filesfor attaching as related data to a tertiary mark (i.e., being used toimplement a UAC command 400-1 uac), which for instance could be issuedto a PA and Music System 400-J5 entertainment device for creating anaudio output via a text-to-speech conversion, or to a Video Display400-J6 entertainment device for visual posting. The SQL selectstatements (such as “SELECT FROM database WHERE sport=ice hockey”) areideal as automatically generated Data Source Inclusion Filters for alsoattaching as related data to a tertiary mark (i.e., being used toimplement a UAC command 400-1 uac), which for instance could be issuedto a PA and Music System 400-J5 entertainment device for playing music,where the SQL select statement is then used by the entertainment device(or its associated wrapper 400-1 b) to query a database of music toretrieve one or more possible Data Files for output. As will also beunderstood by a careful reading of the present application, the issuanceof an ongoing stream of tertiary marks (representing universal actioncommands 400-1 uac) with associated related datum describing currenthappenings (as derived from either observation 100-4 m, 200-4 m marks orintegrated events) can be used as a Data stream for creating an on-goingcommentary of the performance to be output on various audio or videoentertainment devices.

CONCLUSION AND RAMIFICATIONS

Thus the reader will see that the present invention teaches its objectsand advantages as summarized in the opening of the specificationincluding:

-   -   Providing a universal interface module that can be connected to        a scoreboard console either at the juncture between the        console's keyboard and its internal processor, or between its        internal processor and the scoreboard itself where this        interface module provides both training and live modes. In the        training mode, the module is capable of recording various        signals that are either supplied by the keyboard to the console,        or by the console to the scoreboard, representing the entire        range of possible low-level commands performed by the        manufacturer's scoreboard console. In the live mode, the module        is capable of receiving high-level commands from a third-party        scorekeeping console which are then translated into the        equivalent low-level commands and transmitted either to the        scoreboard console via the keyboard, as if they were being        directly entered by the scoreboard console operator, or        transmitted to the real scoreboard, as if they were being        generated by the manufacturer's console;    -   Providing a universal interface module that is cable of        connecting a third-party scorekeeping console to a scoreboard        without requiring a training mode. In this case, the        scorekeeping console maintains an internal virtual scoreboard        that the interface module translates into the necessary signals        for updating the real scoreboard;    -   Providing a scorekeeping system that allows the game officials        in combination with a performance content generation system that        comprises an object tracking system, to perform all official        scorekeeping tasks including at least the operation of the game        clock and the entry of shots, goals and penalties        information—thus providing the option of eliminating the        traditional scorekeeper;    -   Providing apparatus and data translation methods for receiving        human and machine observations from the combination of a        scorekeeping system and a performance content generation system        as related at least to a sports performance such as a game, and        then automatically determining when and which connected        entertainment devices should be commanded to take which specific        actions; where the entertainment devices at least include music        and announcement systems, video displays, scoring indication        lamps, primary arena lighting systems, laser show and secondary        lighting systems as well as dynamic advertising display boards;        and    -   Providing for the implementation of the preferred automatic        entertainment processor using the prior taught session processor        and its various features.

As will be apparent to those familiar with the various marketplaces andtechnologies discussed herein, portions of the present invention areuseful individually or in lesser combinations than the entire scope ofthe aforementioned objects and advantages. Furthermore, while theapparatus and methods are exemplified with respect to the sport of icehockey, as will be obvious to the skilled reader, there are norestrictions on the application of the present teachings, whether toother sports, music, theatre, education, security, business, etc., andin general to any ongoing measurable activities, real, virtual,abstract, animate or inanimate, without limitation. The lack of a needor use in other such applications for a scorekeeping system does notreduce the benefits provided by using a performance content generationsystem in combination with an automatic entertainment system.

Furthermore, as will be obvious to those skilled in the arts of signagecontrol systems, the present teachings in general relate to any “closed”signage control system that includes both a manually operated consolefor accepting signage control signals via a keyboard and then also forgenerating translated control signals for output to one or more signs.While this type of signage control system is prevalent in sports, thepresent invention should not be limited to sporting applications, butrather it applies to the situation of allowing a third party system tocontrol signage by issuing appropriate signals through an interfacemodule that connects with the originally supplied console for operatingthe signage. This universal interface apparatus and method areespecially useful when the third party system not only includes data fordisplay on the signage, but also is capable of receiving inputsautomatically sensed from the surrounding environment, thus providingfor an automatic means of changing signage display based at least inpart upon the uncontrolled dynamic changes in the environment that thesignage is meant to service.

From the foregoing detailed description of the present invention, itwill be apparent that the invention has a number of advantages, some ofwhich have been described herein and others of which are inherent in theinvention. Also, it will be apparent that modifications can be made tothe present invention without departing from the teachings of theinvention, including the sub-division of useful parts for lesserapparatus and methods, still wholly encompassing one or more ideasherein taught.

It is understood that the examples and embodiments that are describedherein are for illustrative purposes only and that various modificationsand changes in light thereof will be suggested to persons skilled in theart and are to be included with the spirit and purview of thisapplication and scope of the appended claims and their full scope ofequivalents. For example, the order of processing of information ispreferred and sufficient but can be adjusted and rearranged withacceptable tradeoffs. Stages and steps that are depicted in series mayinstead occur in parallel. Stages or steps may be skipped, other stagesand steps may be added, etc. Also, for example, the softwaredescriptions, encapsulations, attributes and methods suggested andpreferred by the present inventors to best embody the taught apparatusand methods are a hybrid of well understood object oriented concepts.Other software modalities are sufficiently equivalent to alternatelyembody the taught apparatus and methods without departing from theteachings herein. Furthermore, software modules and objects could becombined or broken apart, associations between objects could be varied,and attributes could be shifted between objects or converted into newobjects. Existing objects could be converted into attributes or methodswithin other existing or new objects, etc.

Accordingly, the scope of the invention is only to be limited asnecessitated by the accompanying claims.

1. A system for use during a live sporting event for controlling atleast one information output device comprising: a first input device foraccepting game information related to the live sporting event; aperformance content generation sub-system, the performance contentgeneration sub-system being operable to automatically generate gameinformation based on the live sporting event; and an output sub-system,the output sub-system being operable to receive game information fromthe first input device and the performance content generation sub-systemand to send commands to the at least one information output device, thecommands consisting of at least one of: a) at least part of the gameinformation; b) electronic instructions based at least in part on thegame information.
 2. The system of claim 1 wherein the at least oneinformation output device is a scoreboard.
 3. The system of claim 2further comprising a universal scoreboard interface device, theuniversal scoreboard interface device being operable to: a. receivecommands from the output sub-system; b. translate the commands intosignals recognizable by the scoreboard; c. provide the signals to thescoreboard.
 4. The system of claim 3 wherein the scoreboard includes agame clock that starts and stops to indicate official time.
 5. Thesystem of claim 4 wherein the game information includes clock statechanges.
 6. The system of claim 4 wherein the first input device isoperable by a game official.
 7. The system of claim 6 further comprisinga second input device, the second input device being operable totransmit clock state changes to the output sub-system.
 8. The system ofclaim 7 wherein the second input device is a wireless clicker.
 9. Thesystem of claim 4 wherein the game information generated by theperformance content generation sub-system comprises clock state changes.10. The system of claim 9 wherein clock start indications are generatedby the performance content generation sub-system based in part on themotion of a player or a game object of the live sporting event.
 11. Thesystem of claim 10 wherein the live sporting event is an ice hockey gameand the game object is a ice hockey puck.
 12. The system of claim 2wherein the game information generated by the performance contentgeneration sub-system comprises scoring information.
 13. The system ofclaim 1 wherein the output device is a primary lighting system forproviding general illumination of the event whose color or luminosity isadjusted in response to the commands from the output sub-system.
 14. Thesystem of claim 13 wherein the primary lighting system uses LED lightscapable of creating multiple colors, and wherein the commands from theoutput sub-system directs the colors to change.
 15. The system of claim1 wherein the output device is a scoring indication lamp whose color orluminosity is adjusted in response to the commands from the outputsub-system.
 16. The system of claim 1 wherein the output device is anaudio system that is directed to output words or music in response tocommands from the output sub-system.
 17. The system of claim 1 whereinthe output device is a video or a video and audio output device that isdirected to output video or video and audio in response to the commandsfrom the output sub-system.
 18. The system of claim 17 wherein the videoor video and audio output device is a scoreboard.
 19. The system ofclaim 17 wherein the video or video and audio output device is a dynamicadvertising display.
 20. The system of claim 19 wherein the dynamicadvertising display is affixed to structures facing inward toward aperformance area of the event.
 21. The system of claim 19 wherein thedynamic advertising display is flexible for conforming to curvedsurfaces and comprises at least a flexible display technology.
 22. Thesystem of claim 21 wherein the dynamic advertising display furthercomprises a base illumination layer to enhance the visibility of theflexible display technology.
 23. The system of claim 19 wherein thedynamic advertising display further comprises a covering layer toprotect its integrity.
 24. The system of claim 19 wherein the dynamicadvertising display further comprises a water-proofing layer to protectits integrity.
 25. The system of claim 1 wherein the output device is asecondary lighting system for providing controlled illumination oflimited areas whose color, luminosity or illumination pattern isadjusted in response to the commands from the output sub-system.
 26. Thesystem of claim 25 wherein the secondary lighting system is a laserprojection device and where the system directs the projection of a teamlogo or symbol in response to commands from the output sub-system. 27.The system of claim 1 wherein the game information generated by theperformance content generation sub-system comprises video or audio ofthe live sporting event.
 28. A system for controlling a display outputon a sports scoreboard at an event, comprising: a third-party device,the third party device comprising: a processor; a user interface forinputting game information; a communication sub-system; wherein theprocessor is operable to generate a virtual scoreboard from the gameinformation; and a universal scoreboard interface comprising: aprocessor; a communication sub-system the universal scoreboard interfacebeing operable to: receive the virtual scoreboard from the third-partydevice; translate the virtual scoreboard into signals recognizable bythe scoreboard; provide the signals to the scoreboard.
 29. The system ofclaim 28 wherein the scoreboard includes a game clock that starts andstops to indicate official time.
 30. The system of claim 29 wherein theuser interface includes one or more buttons for indicating clock statechanges.
 31. The system of claim 29 wherein a game official uses aportable device in communication with the system for indicating clockstate changes.
 32. The system of claim 31 wherein the portable device isa wireless clicker.
 33. The system of claim 28 further including aperformance content generation sub-system, the performance contentgeneration sub-system being operable to automatically generate gameinformation based on the live sporting event; wherein the gameinformation is used as an input to one of the third party device and theuniversal scoreboard interface.
 34. The system of claim 33 wherein thescoreboard includes a game clock that starts and stops to indicateofficial time and wherein the game information generated by theperformance content generation sub-system includes clock state changes.35. The system of claim 34 wherein the clock start indications aregenerated by the performance content generation sub-system based on themotion of a player or a game object.
 36. The system of claim 35 whereinthe sporting event is an ice hockey game and the game object is a icehockey puck.
 37. The system of claim 33 wherein the game informationgenerated by the performance content generation sub-system includesscoring information.
 38. The system of claim 28 that further controlsone or more output devices other than the scoreboard.
 39. The system ofclaim 38 wherein at least one of the one or more output devices is aprimary lighting system for providing general illumination of the eventwhose color or luminosity is adjusted in response to the gameinformation.
 40. The system of claim 39 wherein the primary lightingsystem uses LED lights capable of creating multiple colors, and whereinthe system directs the colors to change in response to game information.41. The system of claim 38 wherein at least one of the one or moreoutput devices is an audio system that is directed to output words ormusic in response to the game information.
 42. The system of claim 38wherein at least one of the one or more output devices is a video orvideo and audio system that is directed to output video or video andaudio in response to the game information.
 43. The system of claim 42wherein the video or video and audio output device is useable as ascoreboard.
 44. The system of claim 42 wherein the video output deviceis a dynamic advertising display.
 45. The system of claim 44 wherein thedynamic advertising display is affixed to structures facing inwardtoward the performance area of the sporting event.
 46. The system ofclaim 44 wherein the dynamic advertising display is flexible foradditionally conforming to curved surfaces and comprises at least aflexible display technology.
 47. The system of claim 46 wherein thedynamic advertising display further comprises a base illumination layerto enhance the visibility of the flexible display technology.
 48. Thesystem of claim 44 wherein the dynamic advertising display furthercomprises a covering layer to protect its integrity.
 49. The system ofclaim 44 wherein the dynamic advertising display further comprises awater-proofing layer to protect its integrity.
 50. The system of claim38 wherein at least one of the one or more output devices is a secondarylighting system for providing controlled illumination of limited areaswhose color, luminosity or illumination pattern is adjusted in responseto the game information.
 51. The system of claim 50 wherein thesecondary lighting system is a laser projection device and wherein thesystem directs the projection of a team logo or symbol in response tothe game information.
 52. The system of claim 38 wherein the gameinformation generated by the performance content generation sub-systemcomprises video or audio of the ongoing event.
 53. The system of claim33 that further controls one or more output devices other than thescoreboard.
 54. The system of claim 53 wherein at least one of the oneor more output devices is a primary lighting system for providinggeneral illumination of the event whose color or luminosity is adjustedin response to the game information.
 55. The system of claim 54 whereinthe primary lighting system uses LED lights capable of creating multiplecolors, and wherein the system directs the colors to change in responseto game information.
 56. The system of claim 53 wherein at least one ofthe one or more output devices is a scoring indication lamp whose coloror luminosity is adjusted in response to the game information.
 57. Thesystem of claim 53 wherein at least one of the one or more outputdevices is an audio system that is directed to output words or music inresponse to the game information.
 58. The system of claim 53 wherein atleast one of the one or more output devices is a video or video andaudio system that is directed to output video or video and audio inresponse to the game information.
 59. The system of claim 58 wherein thevideo or video and audio output device is useable as a scoreboard. 60.The system of claim 58 wherein the video output device is a dynamicadvertising display.
 61. The system of claim 60 wherein the dynamicadvertising display is affixed to structures facing inward toward theperformance area of the sporting event.
 62. The system of claim 60wherein the dynamic advertising display is flexible for additionallyconforming to curved surfaces and comprises at least a flexible displaytechnology.
 63. The system of claim 62 wherein the dynamic advertisingdisplay further comprises a base illumination layer to enhance thevisibility of the flexible display technology.
 64. The system of claim60 wherein the dynamic advertising display further comprises a covingprotective layer to protect its integrity.
 65. The system of claim 60wherein the dynamic advertising display further comprises awater-proofing layer to protect its integrity.
 66. The system of claim33 wherein the output device is a secondary lighting system for provingcontrolled illumination of limited areas whose color, luminosity orillumination pattern is adjusted in response to the game information.67. The system of claim 66 wherein the secondary lighting system is alaser projection device and where the system directs the projection of ateam logo or symbol in response to the game information.
 68. The systemof claim 33 wherein the game information generated by the performancecontent generation sub-system comprises video or audio of the ongoingevent.
 69. An apparatus for translating virtual scoreboard indicationsfrom a third party device into physical signals capable of controlling ascoreboard, comprising: Inputs for accepting the virtual indicationsfrom the third party device for display on the scoreboard, A processingelement for converting the virtual indications into physical controlsignals recognizable to the scoreboard, and Outputs for relaying thephysical signals to the scoreboard.
 70. The apparatus of claim 69wherein the inputs and processing element are reconfigurable to acceptand process indications from two or more distinct third party devices.71. The apparatus of claim 69 wherein the processing element and theoutputs are reconfigurable to convert the virtual indications intophysical control signals recognizable by two or more distinctscoreboards.
 72. The apparatus of claim 69 wherein the scoreboardincludes a game clock that starts and stops to indicate official time.73. The apparatus of claim 72 wherein the third party device acceptsgame information from a scorekeeper or game official, including clockstate changes.
 74. The apparatus of claim 72 wherein the apparatusdirectly accepts game information from a scorekeeper or game official,including clock state changes.
 75. The apparatus of claim 69 that is incommunication with a wireless clicker for accepting game information.76. The apparatus of claim 72 that is in communication with a wirelessclicker for accepting game clock state changes.
 77. The apparatus ofclaim 69 that is in communication with a performance content generationsub-system capable of generating game information based on a livesporting event, the game information being used as an input for one of:the apparatus and the third party device.
 78. The system of claim 77wherein the game information generated by the performance contentgeneration sub-system includes clock state changes.
 79. The system ofclaim 78 wherein the clock start indications are generated by theperformance content generation sub-system based at least in part on themotion of a player or a game object.